diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..52373fe24473b1aa44333d318f578ae6bf04b49b 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000000000000000000000000000000000000..f860919cd5a5377c0bc9d590ea9d1b8743475ca5
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,29 @@
+GLM-5.3 License
+
+Copyright (c) 2026 Z.AI
+
+Permission is hereby granted, free of charge, to any person or entity (the "Licensee") obtaining a copy of this software — including the model weights, parameters, configuration files, inference and training code, and associated documentation (collectively, the "Software") — to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software; to run, deploy, fine-tune, or otherwise modify the Software and create derivative works from it; and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. The Licensee's use of the Software must comply with applicable laws and regulations.
+
+2. "Model as a Service" means giving a third party access to language model inference or fine-tuning (e.g., via API) in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data. This does not include (a) end-user products with model capabilities solely embedded within specific features or harnesses, or (b) mere relaying of requests to models hosted by others.
+If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 10 billion US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must pass Z.AI's security review before using the Software or its derivative works for any commercial purpose. The scope and method of the security review shall be reasonably determined by Z.AI.
+
+3. THE SOFTWARE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL Z.AI OR ITS AFFILIATES OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+For any questions regarding this license, please contact glmlicense@z.ai.
+
+-----
+
+版权所有 (c) 2026 Z.AI
+
+特此免费授予任何获得本软件副本的个人或实体（"被许可方"）——包括模型权重、参数、配置文件、推理和训练代码及相关文档（统称"软件"）——不受限制地处理本软件的权利，包括但不限于：使用、复制、修改、合并、发布、分发、再许可和/或销售软件副本；运行、部署、微调或以其他方式修改软件并创建衍生作品；以及允许获得软件的其他人行使上述权利，但须遵守以下条件：
+
+1. 上述版权声明和本许可声明应包含在软件的所有副本或实质性部分中。被许可方对软件的使用必须符合适用法律法规。
+
+2. "模型即服务"指以允许第三方对输入、参数或训练数据行使实质性控制的方式，向第三方提供语言模型推理或微调服务（如通过API）。不包括：(a) 模型能力仅嵌入特定功能或框架中的终端用户产品，或(b) 单纯转发请求至他人托管的模型。
+若被许可方或其关联方运营"模型即服务"业务，且被许可方及关联方在任意连续12个月内累计总收入超过100亿美元（或等值其他货币），则被许可方在使用软件或其衍生作品进行任何商业用途之前，须通过Z.AI的安全审查。安全审查的范围和方式由Z.AI合理确定。
+
+3. 软件及其任何输出和结果均按"现状"提供，不附带任何形式的保证，无论是明示还是暗示，包括但不限于适销性、特定用途适用性和不侵权的保证。在任何情况下，Z.AI或其关联方或版权持有人均不对任何索赔、损害或其他责任承担责任，无论该责任是基于合同、侵权或其他方式，因软件或使用软件而产生或与之相关。
+
+如对本许可有任何疑问，请联系 glmlicense@z.ai。
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..4d6c30589866ccc3d39ba484849fcd8cd2333f12
--- /dev/null
+++ b/README.md
@@ -0,0 +1,97 @@
+---
+license: other
+license_name: glm-5.3
+license_link: LICENSE
+base_model: zai-org/GLM-5.3
+base_model_relation: quantized
+tags:
+- mlx
+- apple-silicon
+- glm_moe_dsa
+- mixture-of-experts
+- 8-bit
+pipeline_tag: text-generation
+library_name: mlx
+---
+
+# GLM-5.3-MLX-8bit
+
+MLX (Apple Silicon) build of [**GLM-5.3**](https://huggingface.co/zai-org/GLM-5.3) — 744B-parameter
+`glm_moe_dsa` MoE (256 experts, top-8; MLA with DeepSeek-V3.2-style sparse attention) — quantized
+to **8-bit**.
+
+**These files are modified**: converted from the upstream **bfloat16** release
+([GLM-5.3-BF16](https://huggingface.co/zai-org/GLM-5.3-BF16)) to MLX and quantized; the architecture
+is unchanged. The multi-token-prediction layer (78) is not included.
+
+## Runtime — read this
+
+This checkpoint bundles `glm_moe_dsa.py` (declared via `model_file`) and needs it:
+
+```bash
+pip install -U mlx-lm
+mlx_lm.generate --model pipenetwork/GLM-5.3-MLX-8bit --trust-remote-code --prompt "..." --max-tokens 300
+```
+
+mlx-lm's own `glm_moe_dsa` builds a lightning indexer on all 78 layers, but GLM-5.2/5.3 ship
+indexer weights on 21 (`indexer_types`: the other 57 "shared" layers reuse the previous full layer's
+top-k selection). A strict load of the release fails with 285 missing parameters; `mlx_lm.load`
+loads leniently and leaves those 57 indexers at random initialisation. Prompts up to 2048 tokens are
+unaffected (the indexer is bypassed below `index_topk`); beyond that, 57 layers attend to keys
+chosen by random projections. The bundled runtime implements the schedule as the reference does,
+plus the reference's fp32 indexer scores and router logits and the indexer LayerNorm epsilon.
+Tiny-config parity against `transformers` 5.16 is **4e-7** with the sparse path live, cached decode
+exact; strict loading of this checkpoint reports zero missing and zero unexpected tensors. Details and
+tests: [https://github.com/PipeNetwork/glm53-mlx](https://github.com/PipeNetwork/glm53-mlx).
+
+## Size and what is quantized
+
+**790.2 GB** on disk. RAM: 1 TB+ (two machines).
+
+| group | share of parameters | this build |
+|---|---:|---|
+| routed experts (`switch_mlp`, 75 layers × 256) | 724.8B (97.5%) | 8-bit, group 64 |
+| attention (MLA), shared experts, dense layers 0–2, embeddings, `lm_head` | 18.4B (2.5%) | 8-bit, group 64 |
+| lightning indexer (21 layers), MoE router + correction bias, norms | 0.3B | as stored (bf16 / fp32) |
+
+Source precision: the FP8 release is a lossy derivative of the bf16 one (dequantized FP8 weights
+differ from bf16 by up to 1.6e-2 on values of 0.46 — half an e4m3 step). The ladder row `fp8` is the FP8 release itself measured against bf16: its error is the floor any FP8-sourced build inherits.
+
+## Quality
+
+Two measurements, because at 744B most of the ladder cannot be loaded on a 512 GB machine:
+
+**Per-layer divergence vs bf16** (`scripts/eval_ladder.py`): every decoder layer run in bf16 and in
+each recipe on identical inputs (16,384 tokens of wikitext-2), *teacher-forced* (each layer
+sees bf16 inputs — isolates its own damage) and *free-running* (each recipe feeds itself — what
+inference does). Relative L2 error of the layer output; lower is better.
+
+| recipe | teacher-forced (mean over layers) | free-running (final layer) | cosine (final) |
+|---|---:|---:|---:|
+| 8bit | 0.00685 | 0.13119 | 0.98945 |
+| 6bit | 0.01465 | 0.16736 | 0.98389 |
+| 5bit | 0.02651 | 0.22521 | 0.97272 |
+| 4bit | 0.05161 | 0.35740 | 0.93390 |
+| mixed-4_8bit | 0.02524 | 0.24951 | 0.96710 |
+| mixed-3_6bit | 0.05242 | 0.42380 | 0.90624 |
+| fp8 | 0.01741 | 0.17321 | 0.98320 |
+
+**Perplexity** on wikitext-2 (test), 288,627 tokens in 141 windows of 2048, for the builds
+that fit this machine, scored on identical windows:
+
+| build | size | perplexity [95% CI] |
+|---|---:|---|
+| [4bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-4bit) | 418.6 GB | 2.8636 [2.6681, 3.0714] |
+| [mixed-4_8bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-mixed-4_8bit) | 427.8 GB | 2.7420 [2.5533, 2.9477] |
+| [mixed-3_6bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-mixed-3_6bit) | 332.6 GB | 3.0338 [2.8366, 3.2386] |
+| [REAP25-4bit](https://huggingface.co/pipenetwork/GLM-5.3-REAP25-MLX-4bit) | 316.6 GB | 3.2872 [3.0703, 3.5184] |
+| [REAP37-4bit](https://huggingface.co/pipenetwork/GLM-5.3-REAP37-MLX-4bit) | 267.2 GB | 3.8517 [3.6212, 4.0937] |
+| [REAP50-4bit](https://huggingface.co/pipenetwork/GLM-5.3-REAP50-MLX-4bit) | 214.7 GB | 5.0295 [4.7571, 5.3137] |
+
+**Recommendation.** For a 512 GB Mac, **mixed 4/8-bit** (427.7 GB): perplexity 2.7420, a paired 4.3% better than uniform 4-bit (ratio 0.9575 [0.9537, 0.9612], better on 98.6% of windows) for 9 GB more — the 2.5% of non-expert weights are worth their 8 bits, as on every model we have measured. Uniform 4-bit (418.6 GB) is the fallback when those 9 GB matter. **Mixed 3/6-bit** (332.6 GB) is the 384 GB-class option, at a real cost: 3.0338, +5.9% over 4-bit and +10.6% over mixed 4/8 — it leads the ladder for the first ten layers and then 3-bit expert damage compounds. Among the builds that cannot be run here, the ladder puts 8-bit closest to bfloat16 (free-running error 0.131), then 6-bit (0.167); the **upstream FP8 release scores 0.173, between 6-bit and 5-bit**, which is why these are converted from the bf16 release. 5-bit (0.225) sits just above mixed 4/8 (0.250) at 100 GB more.
+
+Greedy generation (a collapse detector, not a ranking) is coherent on every published build.
+
+## License
+
+[GLM-5.3 license](LICENSE), as the upstream model. Port code: [https://github.com/PipeNetwork/glm53-mlx](https://github.com/PipeNetwork/glm53-mlx).
diff --git a/__pycache__/glm_moe_dsa.cpython-314.pyc b/__pycache__/glm_moe_dsa.cpython-314.pyc
new file mode 100644
index 0000000000000000000000000000000000000000..3e609238754edc27fe3a67b1b22bd129fc4a3b92
Binary files /dev/null and b/__pycache__/glm_moe_dsa.cpython-314.pyc differ
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..d226a6928e6ba040e2eadccaf832c037bc4cd98f
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1,251 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = reasoning_effort if reasoning_effort is defined and reasoning_effort in ['low', 'high'] else 'max' -%}
+{%- if effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+{%- macro tool_references_to_response(refs) -%}
+    {{- '<tool_response><tools>\n' -}}
+    {%- for tr in refs -%}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- elif item is mapping and item.type in ['image', 'image_url', 'video', 'video_url', 'audio', 'audio_url', 'input_audio'] -%}
+                {%- set media_type = item.type | replace('_url', '') | replace('input_', '') -%}
+                {{- "<reminder>You are unable to process this " ~ media_type ~ " because you don't have multi-modal input ability. Try different methods.</reminder>" }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro tool_response(text) -%}
+{{- '<tool_response>' + text + '</tool_response>' -}}
+{%- endmacro -%}
+{%- macro render_tool_response(m) -%}
+{%- if m.content is string -%}
+    {{- tool_response(m.content) -}}
+{%- elif m.content and m.content is not mapping and m.content.0.type == "tool_reference" -%}
+    {{- tool_references_to_response(m.content) -}}
+{%- elif is_list_of_outputs(m) -%}
+    {%- for tr in m.content -%}
+        {%- if tr.output is iterable and tr.output is not string and tr.output is not mapping and tr.output and tr.output.0.type == "tool_reference" -%}
+            {{- tool_references_to_response(tr.output) -}}
+        {%- else -%}
+            {{- tool_response(visible_text(tr.output)) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+    {{- tool_response(visible_text(m.content)) -}}
+{%- endif -%}
+{%- endmacro -%}
+{%- macro id_of(obj) -%}
+    {%- if obj.tool_call_id -%}
+        {{- obj.tool_call_id -}}
+    {%- elif obj.id -%}
+        {{- obj.id -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro is_list_of_outputs(m) -%}
+    {%- if m.content and m.content.0.output is defined -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro has_dup_tool_result_id(lo, hi, target) -%}
+    {%- set ns_cnt = namespace(n=0) -%}
+    {%- for k in range(lo, hi + 1) -%}
+        {%- set m = messages[k] -%}
+        {%- if is_list_of_outputs(m) -%}
+            {%- for entry in m.content -%}
+                {%- if id_of(entry) == target -%}
+                    {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- elif id_of(m) == target -%}
+            {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+        {%- endif -%}
+        {%- if ns_cnt.n > 1 -%}{%- break -%}{%- endif -%}
+    {%- endfor -%}
+    {%- if ns_cnt.n > 1 -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro tc_id_exists(tcs, target) -%}
+    {%- set ns_f = namespace(found=false) -%}
+    {%- for tc in tcs -%}
+        {%- if id_of(tc) == target -%}
+            {%- set ns_f.found = true -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- if ns_f.found -%}1{%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if (not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+    {%- set block_start = loop.index0 -%}
+    {%- set ns_blk = namespace(end=block_start) -%}
+    {%- for j in range(block_start, messages|length) -%}
+        {%- if messages[j].role == 'tool' -%}
+            {%- set ns_blk.end = j -%}
+        {%- else -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns_a = namespace(tool_calls=none) -%}
+    {%- if block_start > 0 and messages[block_start - 1].role == 'assistant' and messages[block_start - 1].tool_calls -%}
+        {%- set ns_a.tool_calls = messages[block_start - 1].tool_calls -%}
+    {%- endif -%}
+    {%- set ns_chk = namespace(can_sort=true) -%}
+    {%- if not ns_a.tool_calls -%}
+        {%- set ns_chk.can_sort = false -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {%- set m = messages[k] -%}
+            {%- if is_list_of_outputs(m) -%}
+                {%- for entry in m.content -%}
+                    {%- set eid = id_of(entry) -%}
+                    {%- if not eid -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif has_dup_tool_result_id(block_start, ns_blk.end, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif not tc_id_exists(ns_a.tool_calls, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- else -%}
+                {%- set tk_id = id_of(m) -%}
+                {%- if not tk_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif has_dup_tool_result_id(block_start, ns_blk.end, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif not tc_id_exists(ns_a.tool_calls, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- for i in range(ns_a.tool_calls | length) -%}
+            {%- set tc_id = id_of(ns_a.tool_calls[i]) -%}
+            {%- if not tc_id -%}
+                {%- set ns_chk.can_sort = false -%}
+            {%- endif -%}
+            {%- for j in range(i + 1, ns_a.tool_calls | length) -%}
+                {%- if id_of(ns_a.tool_calls[j]) == tc_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if ns_chk.can_sort -%}
+        {%- for tc in ns_a.tool_calls -%}
+            {%- set tc_id = id_of(tc) -%}
+            {%- for k in range(block_start, ns_blk.end + 1) -%}
+                {%- set m = messages[k] -%}
+                {%- if is_list_of_outputs(m) -%}
+                    {%- for entry in m.content -%}
+                        {%- set eid = id_of(entry) -%}
+                        {%- if eid == tc_id -%}
+                            {%- if entry.output is iterable and entry.output is not string and entry.output is not mapping and entry.output and entry.output.0.type == "tool_reference" -%}
+                                {{- tool_references_to_response(entry.output) -}}
+                            {%- else -%}
+                                {{- tool_response(visible_text(entry.output)) -}}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- else -%}
+                    {%- set tk_id = id_of(m) -%}
+                    {%- if tk_id == tc_id -%}
+                        {{- render_tool_response(m) -}}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {{- render_tool_response(messages[k]) -}}
+        {%- endfor -%}
+    {%- endif -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think>' -}}
+{%- endif -%}
diff --git a/config.json b/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..c6f843ac8b47e7414c78993906a184b009011aa4
--- /dev/null
+++ b/config.json
@@ -0,0 +1,233 @@
+{
+  "architectures": [
+    "GlmMoeDsaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "ep_size": 1,
+  "first_k_dense_replace": 3,
+  "head_dim": 192,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "index_head_dim": 128,
+  "index_n_heads": 32,
+  "index_share_for_mtp_iteration": true,
+  "index_skip_topk_offset": 3,
+  "index_topk": 2048,
+  "index_topk_freq": 4,
+  "index_topk_pattern": null,
+  "indexer_rope_interleave": true,
+  "indexer_types": [
+    "full",
+    "full",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared",
+    "full",
+    "shared",
+    "shared",
+    "shared"
+  ],
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "kv_lora_rank": 512,
+  "max_position_embeddings": 1048576,
+  "mlp_layer_types": [
+    "dense",
+    "dense",
+    "dense",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse"
+  ],
+  "model_type": "glm_moe_dsa",
+  "moe_intermediate_size": 2048,
+  "moe_layer_freq": 1,
+  "moe_router_dtype": "float32",
+  "n_group": 1,
+  "n_routed_experts": 256,
+  "n_shared_experts": 1,
+  "norm_topk_prob": true,
+  "num_attention_heads": 64,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 78,
+  "num_key_value_heads": 64,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 154820,
+  "pretraining_tp": 1,
+  "q_lora_rank": 2048,
+  "qk_head_dim": 256,
+  "qk_nope_head_dim": 192,
+  "qk_rope_head_dim": 64,
+  "rms_norm_eps": 1e-05,
+  "rope_interleave": true,
+  "rope_parameters": {
+    "rope_theta": 8000000,
+    "rope_type": "default"
+  },
+  "routed_scaling_factor": 2.5,
+  "scoring_func": "sigmoid",
+  "tie_word_embeddings": false,
+  "topk_group": 1,
+  "topk_method": "noaux_tc",
+  "transformers_version": "5.15.0",
+  "use_cache": true,
+  "v_head_dim": 256,
+  "vocab_size": 154880,
+  "quantization": {
+    "group_size": 64,
+    "bits": 8
+  },
+  "quantization_config": {
+    "group_size": 64,
+    "bits": 8
+  },
+  "model_file": "glm_moe_dsa.py"
+}
\ No newline at end of file
diff --git a/generation_config.json b/generation_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..216bbb092d0df0dda1dc4cedd789b92286c4c001
--- /dev/null
+++ b/generation_config.json
@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.12.0"
+}
diff --git a/glm_moe_dsa.py b/glm_moe_dsa.py
new file mode 100644
index 0000000000000000000000000000000000000000..233864a54415a460ab55cf50db2a8e65aaf786ea
--- /dev/null
+++ b/glm_moe_dsa.py
@@ -0,0 +1,703 @@
+# Copyright © 2025 Apple Inc.
+
+import math
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+import mlx.core as mx
+import mlx.nn as nn
+from mlx.nn.layers.distributed import shard_inplace, shard_linear, sum_gradients
+
+from mlx_lm.models.activations import swiglu
+from mlx_lm.models.base import BaseModelArgs, create_attention_mask, scaled_dot_product_attention
+from mlx_lm.models.cache import CacheList, KVCache
+from mlx_lm.models.mla import MultiLinear
+from mlx_lm.models.rope_utils import initialize_rope
+from mlx_lm.models.switch_layers import SwitchGLU
+
+
+@dataclass
+class ModelArgs(BaseModelArgs):
+    model_type: str
+    vocab_size: int
+    hidden_size: int
+    index_head_dim: int
+    index_n_heads: int
+    index_topk: int
+    intermediate_size: int
+    moe_intermediate_size: int
+    num_hidden_layers: int
+    num_attention_heads: int
+    num_key_value_heads: int
+    n_shared_experts: Optional[int]
+    n_routed_experts: Optional[int]
+    routed_scaling_factor: float
+    kv_lora_rank: int
+    q_lora_rank: int
+    qk_rope_head_dim: int
+    v_head_dim: int
+    qk_nope_head_dim: int
+    topk_method: str
+    scoring_func: str
+    norm_topk_prob: bool
+    n_group: int
+    topk_group: int
+    num_experts_per_tok: int
+    moe_layer_freq: int
+    first_k_dense_replace: int
+    max_position_embeddings: int
+    rms_norm_eps: float
+    rope_parameters: Dict
+    attention_bias: bool
+    rope_scaling: Dict = None
+    rope_theta: Optional[float] = None
+    indexer_rope_interleave: bool = True
+    rope_interleave: bool = True
+    indexer_types: Optional[list] = None
+    index_topk_freq: int = 1
+    index_skip_topk_offset: int = 0
+    index_topk_pattern: Optional[Any] = None
+    num_nextn_predict_layers: int = 0
+    mlp_layer_types: Optional[list] = None
+
+    def __post_init__(self):
+        self.rope_scaling = self.rope_parameters
+        self.rope_theta = self.rope_parameters["rope_theta"]
+        if self.indexer_types is None:
+            # Reference schedule: a layer runs its own indexer ("full") iff
+            # max(i - offset + 1, 0) % freq == 0; the others ("shared") reuse the most recent
+            # full layer's top-k selection and carry no indexer weights.
+            f, o = self.index_topk_freq, self.index_skip_topk_offset
+            self.indexer_types = [
+                "full" if max(i - o + 1, 0) % f == 0 else "shared"
+                for i in range(self.num_hidden_layers)
+            ]
+
+
+
+class Indexer(nn.Module):
+    def __init__(self, args: ModelArgs):
+        super().__init__()
+        self.dim = args.hidden_size
+        self.n_heads = args.index_n_heads
+        self.head_dim = args.index_head_dim
+        self.rope_head_dim = args.qk_rope_head_dim
+        self.index_topk = args.index_topk
+        self.q_lora_rank = args.q_lora_rank
+        self.wq_b = nn.Linear(
+            self.q_lora_rank, self.n_heads * self.head_dim, bias=False
+        )
+        self.wk = nn.Linear(self.dim, self.head_dim, bias=False)
+        self.k_norm = nn.LayerNorm(self.head_dim, eps=1e-6)
+        self.weights_proj = nn.Linear(self.dim, self.n_heads, bias=False)
+        self.softmax_scale = self.head_dim**-0.5
+        self.rope = initialize_rope(
+            dims=args.qk_rope_head_dim,
+            base=args.rope_theta,
+            traditional=args.indexer_rope_interleave,
+            max_position_embeddings=args.max_position_embeddings,
+            scaling_config=args.rope_scaling,
+        )
+
+    def __call__(
+        self,
+        x: mx.array,
+        qr: mx.array,
+        mask: Optional[mx.array],
+        cache: Optional[Any] = None,
+    ):
+        # Computes top_k indices for attention
+        b, s, _ = x.shape
+        q = self.wq_b(qr)
+        q = q.reshape(b, s, self.n_heads, self.head_dim).swapaxes(1, 2)
+        k = self.wk(x)
+        k = self.k_norm(k)
+        k = mx.reshape(k, (b, 1, s, self.head_dim))
+
+        offset = cache.offset if cache is not None else 0
+
+        q = self.rope(q, offset=offset)
+        k = self.rope(k, offset=offset)
+
+        if cache is not None:
+            k, _ = cache.update_and_fetch(k, mx.zeros([b, 1, s, 0]))
+        if k.shape[2] <= self.index_topk:
+            return None
+        # Scores in float32, as the reference (which also keeps weights_proj in fp32): in bf16
+        # keys at the top-k boundary flip.
+        scores = q.astype(mx.float32) @ k.astype(mx.float32).swapaxes(-1, -2)
+        scores = mx.maximum(scores, 0)
+        weights = self.weights_proj(x).astype(mx.float32) * (self.n_heads**-0.5 * self.softmax_scale)
+        weights = weights.swapaxes(-1, -2)[..., None]
+        scores = scores * weights
+        scores = scores.sum(axis=1, keepdims=True)
+        if mask is not None:
+            scores = mx.where(mask, scores, -float("inf"))
+        return mx.argpartition(scores, kth=-self.index_topk, axis=-1)[
+            ..., -self.index_topk :
+        ]
+
+
+class DeepseekV32Attention(nn.Module):
+    def __init__(self, config: ModelArgs, layer_idx: int = 0):
+        super().__init__()
+        self.config = config
+        self.hidden_size = config.hidden_size
+        self.num_heads = config.num_attention_heads
+        self.max_position_embeddings = config.max_position_embeddings
+        self.rope_theta = config.rope_theta
+        self.q_lora_rank = config.q_lora_rank
+        self.qk_rope_head_dim = config.qk_rope_head_dim
+        self.kv_lora_rank = config.kv_lora_rank
+        self.v_head_dim = config.v_head_dim
+        self.qk_nope_head_dim = config.qk_nope_head_dim
+        self.q_head_dim = config.qk_nope_head_dim + config.qk_rope_head_dim
+
+        self.scale = self.q_head_dim**-0.5
+
+        self.q_a_proj = nn.Linear(
+            self.hidden_size, self.q_lora_rank, bias=config.attention_bias
+        )
+        self.q_a_layernorm = nn.RMSNorm(self.q_lora_rank, eps=1e-6)
+        self.q_b_proj = nn.Linear(
+            self.q_lora_rank, self.num_heads * self.q_head_dim, bias=False
+        )
+
+        self.kv_a_proj_with_mqa = nn.Linear(
+            self.hidden_size,
+            self.kv_lora_rank + self.qk_rope_head_dim,
+            bias=config.attention_bias,
+        )
+        self.kv_a_layernorm = nn.RMSNorm(self.kv_lora_rank, eps=1e-6)
+        self.embed_q = MultiLinear(
+            self.qk_nope_head_dim, self.kv_lora_rank, self.num_heads
+        )
+        self.unembed_out = MultiLinear(
+            self.kv_lora_rank, self.v_head_dim, self.num_heads
+        )
+
+        self.o_proj = nn.Linear(
+            self.num_heads * self.v_head_dim,
+            self.hidden_size,
+            bias=config.attention_bias,
+        )
+
+        if self.config.rope_scaling is not None:
+            mscale_all_dim = self.config.rope_scaling.get("mscale_all_dim", 0)
+            if mscale_all_dim:
+                scaling_factor = self.config.rope_scaling["factor"]
+                if scaling_factor > 1:
+                    s = 0.1 * mscale_all_dim * math.log(scaling_factor) + 1.0
+                    self.scale = self.scale * s * s
+
+        # "shared" layers carry no indexer weights: they reuse the previous full layer's top-k.
+        self.indexer = Indexer(config) if config.indexer_types[layer_idx] == "full" else None
+        self.rope = initialize_rope(
+            dims=self.qk_rope_head_dim,
+            base=self.rope_theta,
+            traditional=True,
+            max_position_embeddings=self.max_position_embeddings,
+            scaling_config=self.config.rope_scaling,
+        )
+
+    def __call__(
+        self,
+        x: mx.array,
+        mask: Optional[mx.array] = None,
+        cache: Optional[Any] = None,
+        prev_topk_indices: Optional[mx.array] = None,
+    ):
+        B, L, D = x.shape
+
+        qr = self.q_a_layernorm(self.q_a_proj(x))
+        q = self.q_b_proj(qr)
+
+        q = q.reshape(B, L, self.num_heads, self.q_head_dim).transpose(0, 2, 1, 3)
+        q_nope, q_pe = mx.split(q, [self.qk_nope_head_dim], axis=-1)
+        compressed_kv = self.kv_a_proj_with_mqa(x)
+        compressed_kv, k_pe = mx.split(compressed_kv, [self.kv_lora_rank], axis=-1)
+        k_pe = k_pe.reshape(B, L, 1, self.qk_rope_head_dim).transpose(0, 2, 1, 3)
+        kv_latent = self.kv_a_layernorm(compressed_kv)
+
+        offset = cache[0].offset if cache is not None else 0
+        q_pe = self.rope(q_pe, offset)
+        k_pe = self.rope(k_pe, offset)
+
+        kv_latent = mx.expand_dims(kv_latent, axis=1)
+
+        if cache is not None:
+            kv_latent, k_pe = cache[0].update_and_fetch(kv_latent, k_pe)
+        else:
+            cache = [None] * 2
+
+        if self.indexer is not None:
+            topk_indices = self.indexer(x, qr, mask, cache=cache[1])
+        else:
+            topk_indices = prev_topk_indices
+        if topk_indices is not None:
+            if L == 1:
+                idx = topk_indices[:, :, 0, :, None]
+                kv_latent = mx.take_along_axis(
+                    kv_latent,
+                    mx.broadcast_to(idx, idx.shape[:-1] + (kv_latent.shape[-1],)),
+                    axis=2,
+                )
+                k_pe = mx.take_along_axis(
+                    k_pe,
+                    mx.broadcast_to(idx, idx.shape[:-1] + (k_pe.shape[-1],)),
+                    axis=2,
+                )
+                if mask is not None:
+                    mask = mx.take_along_axis(mask, topk_indices, axis=-1)
+            else:
+                shape = list(topk_indices.shape)
+                shape[-1] = kv_latent.shape[2]
+                sparse_mask = mx.zeros(shape, dtype=mx.bool_)
+                sparse_mask = mx.put_along_axis(
+                    sparse_mask, topk_indices, mx.array(True), axis=-1
+                )
+                if mask is not None:
+                    sparse_mask = sparse_mask & mask
+                mask = sparse_mask
+        # Ensure the indexer cache is evaluated even if the topk_indices are unused
+        # to keep the graph from getting too large
+        if self.indexer is not None and cache is not None and cache[0] is not None:
+            cache[0].keys = mx.depends(cache[0].keys, (cache[1].keys, cache[1].values))
+
+        pe_scores = (q_pe * self.scale) @ k_pe.swapaxes(-1, -2)
+        if mask is not None:
+            pe_scores = mx.where(
+                mask,
+                pe_scores,
+                mx.array(mx.finfo(pe_scores.dtype).min, pe_scores.dtype),
+            )
+
+        if L == 1:
+            q_nope = self.embed_q(q_nope)
+            k = v = kv_latent
+        else:
+            k = self.embed_q(kv_latent, transpose=False)
+            v = self.unembed_out(kv_latent)
+
+        output = scaled_dot_product_attention(
+            q_nope, k, v, cache=cache, scale=self.scale, mask=pe_scores
+        )
+        if L == 1:
+            output = self.unembed_out(output)
+
+        output = output.transpose(0, 2, 1, 3).reshape(B, L, -1)
+        return self.o_proj(output), topk_indices
+
+
+class DeepseekV32MLP(nn.Module):
+    def __init__(
+        self, config: ModelArgs, hidden_size: int = None, intermediate_size: int = None
+    ):
+        super().__init__()
+        self.config = config
+        self.hidden_size = config.hidden_size if hidden_size is None else hidden_size
+        self.intermediate_size = (
+            config.intermediate_size if intermediate_size is None else intermediate_size
+        )
+
+        self.gate_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.up_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.down_proj = nn.Linear(self.intermediate_size, self.hidden_size, bias=False)
+
+    def __call__(self, x):
+        down_proj = self.down_proj(swiglu(self.gate_proj(x), self.up_proj(x)))
+        return down_proj
+
+
+@mx.compile
+def group_expert_select(
+    gates,
+    e_score_correction_bias,
+    top_k,
+    n_group,
+    topk_group,
+    routed_scaling_factor,
+    norm_topk_prob,
+):
+
+    scores = mx.sigmoid(gates.astype(mx.float32))
+    orig_scores = scores
+    scores = scores + e_score_correction_bias
+    if n_group > 1:
+        scores = mx.unflatten(scores, axis=-1, shape=(n_group, -1))
+        group_scores = mx.topk(scores, 2, axis=-1).sum(axis=-1, keepdims=True)
+        k = n_group - topk_group
+        group_idx = mx.argpartition(group_scores, kth=k - 1, axis=-2)[..., :k, :]
+        scores = mx.put_along_axis(
+            scores, mx.stop_gradient(group_idx), mx.array(0.0), axis=-2
+        )
+        scores = mx.flatten(scores, -2, -1)
+
+    k = top_k
+    inds = mx.argpartition(-scores, kth=k - 1, axis=-1)[..., :k]
+    scores = mx.take_along_axis(orig_scores, inds, axis=-1)
+    if top_k > 1 and norm_topk_prob:
+        denominator = scores.sum(axis=-1, keepdims=True)
+        scores = scores / denominator
+    scores = scores * routed_scaling_factor
+
+    return inds, scores
+
+
+class MoEGate(nn.Module):
+    def __init__(self, config: ModelArgs):
+        super().__init__()
+        self.config = config
+        self.top_k = config.num_experts_per_tok
+        self.norm_topk_prob = config.norm_topk_prob
+        self.n_routed_experts = config.n_routed_experts
+        self.routed_scaling_factor = config.routed_scaling_factor
+        self.n_group = config.n_group
+        self.topk_group = config.topk_group
+        self.weight = mx.zeros((self.n_routed_experts, config.hidden_size))
+        self.e_score_correction_bias = mx.zeros((self.n_routed_experts,))
+        assert config.topk_method == "noaux_tc", "Unsupported topk method."
+
+    def __call__(self, x):
+        return group_expert_select(
+            x.astype(mx.float32) @ self.weight.astype(mx.float32).T,
+            self.e_score_correction_bias,
+            self.top_k,
+            self.n_group,
+            self.topk_group,
+            self.routed_scaling_factor,
+            self.norm_topk_prob,
+        )
+
+
+class DeepseekV32MoE(nn.Module):
+    def __init__(self, config: ModelArgs):
+        super().__init__()
+        self.config = config
+        self.num_experts_per_tok = config.num_experts_per_tok
+        self.switch_mlp = SwitchGLU(
+            config.hidden_size,
+            config.moe_intermediate_size,
+            config.n_routed_experts,
+        )
+
+        self.gate = MoEGate(config)
+        if config.n_shared_experts is not None:
+            intermediate_size = config.moe_intermediate_size * config.n_shared_experts
+            self.shared_experts = DeepseekV32MLP(
+                config=config, intermediate_size=intermediate_size
+            )
+
+        self.sharding_group = None
+
+    def __call__(self, x):
+        if self.sharding_group is not None:
+            x = sum_gradients(self.sharding_group)(x)
+
+        inds, scores = self.gate(x)
+        y = self.switch_mlp(x, inds)
+        y = (y * scores[..., None]).sum(axis=-2).astype(y.dtype)
+        if self.config.n_shared_experts is not None:
+            y = y + self.shared_experts(x)
+
+        if self.sharding_group is not None:
+            y = mx.distributed.all_sum(y, group=self.sharding_group)
+
+        return y
+
+
+class DeepseekV32DecoderLayer(nn.Module):
+    def __init__(self, config: ModelArgs, layer_idx: int):
+        super().__init__()
+        self.self_attn = DeepseekV32Attention(config, layer_idx)
+        self.mlp = (
+            DeepseekV32MoE(config)
+            if (
+                config.n_routed_experts is not None
+                and layer_idx >= config.first_k_dense_replace
+                and layer_idx % config.moe_layer_freq == 0
+            )
+            else DeepseekV32MLP(config)
+        )
+        self.input_layernorm = nn.RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.post_attention_layernorm = nn.RMSNorm(
+            config.hidden_size, eps=config.rms_norm_eps
+        )
+
+    def __call__(
+        self,
+        x: mx.array,
+        mask: Optional[mx.array] = None,
+        cache: Optional[Any] = None,
+        prev_topk_indices: Optional[mx.array] = None,
+    ):
+        r, topk_indices = self.self_attn(self.input_layernorm(x), mask, cache, prev_topk_indices)
+        h = x + r
+        r = self.mlp(self.post_attention_layernorm(h))
+        return h + r, topk_indices
+
+
+class DeepseekV32Model(nn.Module):
+    def __init__(self, config: ModelArgs):
+        super().__init__()
+        self.vocab_size = config.vocab_size
+        self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size)
+        self.layers = [
+            DeepseekV32DecoderLayer(config, idx)
+            for idx in range(config.num_hidden_layers)
+        ]
+        self.start_idx = 0
+        self.end_idx = len(self.layers)
+        self.num_layers = self.end_idx
+
+        self.norm = nn.RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.pipeline_rank = 0
+        self.pipeline_size = 1
+
+    def pipeline(self, group):
+        # Split layers in reverse so rank=0 gets the last layers and
+        # rank=pipeline_size-1 gets the first
+        self.pipeline_rank = group.rank()
+        self.pipeline_size = group.size()
+        layers_per_rank = len(self.layers) // self.pipeline_size
+        extra = len(self.layers) - layers_per_rank * self.pipeline_size
+        if self.pipeline_rank < extra:
+            layers_per_rank += 1
+        self.start_idx = (self.pipeline_size - self.pipeline_rank - 1) * layers_per_rank
+        self.end_idx = self.start_idx + layers_per_rank
+        self.layers = self.layers[: self.end_idx]
+        self.layers[: self.start_idx] = [None] * self.start_idx
+        self.num_layers = len(self.layers) - self.start_idx
+
+    def __call__(
+        self,
+        x: mx.array,
+        cache: Optional[Any] = None,
+    ) -> mx.array:
+        h = self.embed_tokens(x)
+
+        pipeline_rank = self.pipeline_rank
+        pipeline_size = self.pipeline_size
+
+        if cache is None:
+            cache = [None] * self.num_layers
+        mask = create_attention_mask(
+            h, cache[0][0] if cache[0] else None, return_array=True
+        )
+
+        # Receive from the previous process in the pipeline
+
+        if pipeline_rank < pipeline_size - 1:
+            h = mx.distributed.recv_like(h, (pipeline_rank + 1))
+
+        # A "shared" layer reuses the top-k selection of the most recent "full" layer.
+        topk = None
+        for i in range(self.num_layers):
+            h, topk = self.layers[self.start_idx + i](h, mask, cache[i], topk)
+
+        # Send to the next process in the pipeline
+        if pipeline_rank != 0:
+            h = mx.distributed.send(h, (pipeline_rank - 1) % pipeline_size)
+            if cache[-1] is not None:
+                cache[-1][0].keys = mx.depends(cache[-1][0].keys, h)
+
+        # Broadcast h while keeping it in the graph
+        if pipeline_size > 1:
+            h = mx.distributed.all_gather(h)[: h.shape[0]]
+
+        return self.norm(h)
+
+
+class Model(nn.Module):
+    def __init__(self, config: ModelArgs):
+        super().__init__()
+        self.args = config
+        self.model_type = config.model_type
+        self.model = DeepseekV32Model(config)
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+
+    def __call__(
+        self,
+        inputs: mx.array,
+        cache: Optional[Any] = None,
+    ):
+        out = self.model(inputs, cache)
+        return self.lm_head(out)
+
+    def sanitize(self, weights):
+        # Remove multi-token prediction layers
+        mpt_layer = self.args.num_hidden_layers
+        new_weights = {}
+        for k, v in weights.items():
+            parts = k.split(".")
+            if len(parts) >= 3 and parts[1] == "layers" and int(parts[2]) >= mpt_layer:
+                continue
+            new_weights[k] = v
+        weights = new_weights
+
+        def dequant(weight, scale_inv):
+            dtype = mx.bfloat16
+            weight = mx.from_fp8(weight, dtype=mx.bfloat16)
+            bs = 128  # block size
+            m, n = weight.shape
+            pad_bottom = (-m) % bs
+            pad_side = (-n) % bs
+            weight = mx.pad(weight, ((0, pad_bottom), (0, pad_side)))
+            weight = weight.reshape(
+                ((m + pad_bottom) // bs, bs, (n + pad_side) // bs, bs)
+            )
+            weight = (weight * scale_inv[:, None, :, None]).reshape(
+                m + pad_bottom, n + pad_side
+            )
+            return weight[:m, :n].astype(dtype)
+
+        # Dequantize
+        new_weights = {}
+        for k, v in weights.items():
+            if "weight_scale_inv" in k:
+                scale_inv = v
+                wk = k.replace("_scale_inv", "")
+                weight = weights[wk]
+                weight = dequant(weight, scale_inv)
+                new_weights[wk] = weight
+            elif k not in new_weights:
+                new_weights[k] = v
+        weights = new_weights
+
+        # Stack experts
+        for l in range(self.args.num_hidden_layers):
+            prefix = f"model.layers.{l}"
+            for n, m in [("w1", "gate_proj"), ("w2", "down_proj"), ("w3", "up_proj")]:
+                for k in ["weight", "scales", "biases"]:
+                    if f"{prefix}.mlp.experts.0.{m}.{k}" in weights:
+                        to_join = [
+                            weights.pop(f"{prefix}.mlp.experts.{e}.{m}.{k}")
+                            for e in range(self.args.n_routed_experts)
+                        ]
+                        weights[f"{prefix}.mlp.switch_mlp.{m}.{k}"] = mx.stack(to_join)
+            prefix = f"model.layers.{l}.self_attn"
+            if f"{prefix}.kv_b_proj.weight" in weights:
+                layer = self.model.layers[l].self_attn.embed_q
+                quantized = f"{prefix}.kv_b_proj.scales" in weights
+                v = weights.pop(f"{prefix}.kv_b_proj.weight")
+                head_dim = self.args.qk_nope_head_dim + self.args.v_head_dim
+
+                if quantized:
+                    dims = self.args.kv_lora_rank
+                    scales = weights.pop(f"{prefix}.kv_b_proj.scales")
+                    biases = weights.pop(f"{prefix}.kv_b_proj.biases")
+                    # Try to infer bits and group size
+                    bits = (v.shape[-1] * 32) // dims
+                    group_size = dims // scales.shape[-1]
+                    v = mx.dequantize(
+                        v, scales, biases, bits=bits, group_size=group_size
+                    )
+                num_heads = self.args.num_attention_heads
+                v = v.reshape(num_heads, head_dim, -1)
+                wk = mx.contiguous(
+                    v[:, : self.args.qk_nope_head_dim, :].swapaxes(-1, -2)
+                )
+                wv = mx.contiguous(v[:, self.args.qk_nope_head_dim :, :])
+                if quantized:
+                    wk, wk_scales, wk_biases = mx.quantize(
+                        wk, bits=bits, group_size=group_size
+                    )
+                    wv, wv_scales, wv_biases = mx.quantize(
+                        wv, bits=bits, group_size=group_size
+                    )
+                    weights[f"{prefix}.embed_q.scales"] = wk_scales
+                    weights[f"{prefix}.unembed_out.scales"] = wv_scales
+                    weights[f"{prefix}.embed_q.biases"] = wk_biases
+                    weights[f"{prefix}.unembed_out.biases"] = wv_biases
+                weights[f"{prefix}.embed_q.weight"] = wk
+                weights[f"{prefix}.unembed_out.weight"] = wv
+
+        return weights
+
+    def shard(self, group: Optional[mx.distributed.Group] = None):
+        group = group or mx.distributed.init()
+        N = group.size()
+        rank = group.rank()
+        for layer in self.model.layers:
+            layer.self_attn.q_b_proj = shard_linear(
+                layer.self_attn.q_b_proj, "all-to-sharded", group=group
+            )
+
+            layer.self_attn.o_proj = shard_linear(
+                layer.self_attn.o_proj, "sharded-to-all", group=group
+            )
+            layer.self_attn.num_heads //= N
+            num_heads = layer.self_attn.num_heads
+            sh = rank * num_heads
+            eh = sh + num_heads
+
+            def shard_heads(w):
+                return w[sh:eh]
+
+            layer.self_attn.embed_q.apply(shard_heads)
+            layer.self_attn.unembed_out.apply(shard_heads)
+
+            # Shard the MLP
+            if isinstance(layer.mlp, DeepseekV32MLP):
+                layer.mlp.gate_proj = shard_linear(
+                    layer.mlp.gate_proj, "all-to-sharded", group=group
+                )
+                layer.mlp.down_proj = shard_linear(
+                    layer.mlp.down_proj, "sharded-to-all", group=group
+                )
+                layer.mlp.up_proj = shard_linear(
+                    layer.mlp.up_proj, "all-to-sharded", group=group
+                )
+
+            # Shard the MoE. Shard in place since the MoE should be responsible
+            # for aggregating the results.
+            else:
+                layer.mlp.sharding_group = group = group
+                shard_inplace(
+                    layer.mlp.shared_experts.gate_proj, "all-to-sharded", group=group
+                )
+                shard_inplace(
+                    layer.mlp.shared_experts.down_proj, "sharded-to-all", group=group
+                )
+                shard_inplace(
+                    layer.mlp.shared_experts.up_proj, "all-to-sharded", group=group
+                )
+                shard_inplace(
+                    layer.mlp.switch_mlp.gate_proj, "all-to-sharded", group=group
+                )
+                shard_inplace(
+                    layer.mlp.switch_mlp.down_proj, "sharded-to-all", group=group
+                )
+                shard_inplace(
+                    layer.mlp.switch_mlp.up_proj, "all-to-sharded", group=group
+                )
+
+    @property
+    def layers(self):
+        return self.model.layers[self.model.start_idx : self.model.end_idx]
+
+    @property
+    def cast_predicate(self):
+        def predicate(k):
+            return "e_score_correction_bias" not in k
+
+        return predicate
+
+    @property
+    def quant_predicate(self):
+        def predicate(path, module):
+            # The lightning indexer (0.03% of parameters, reference keeps weights_proj in fp32) and the
+            # MoE router decide *what* is computed; keep them as stored.
+            if ".indexer." in path or path.endswith("mlp.gate"):
+                return False
+            return True
+
+        return predicate
+
+    def make_cache(self):
+        # Shared layers have no indexer, so no indexer key cache: an empty second KVCache would
+        # trip mlx-lm's `cache.state` evaluation (keys is None) during generation.
+        return [
+            CacheList(KVCache(), KVCache()) if layer.self_attn.indexer is not None else CacheList(KVCache())
+            for layer in self.layers
+        ]
diff --git a/model-00001.safetensors b/model-00001.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5b7adf18476439a41934ccc54b1bbf562521cfe7
--- /dev/null
+++ b/model-00001.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb52ed958f4afddcc845718cc20f60e6b96e03d3d305e0764b6faf4c3a252438
+size 2022126266
diff --git a/model-00002.safetensors b/model-00002.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..053d9a47abb4d421725da9aacbb0dba33243faba
--- /dev/null
+++ b/model-00002.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdb4e5479a2a24ef0d660b9a7bd6c74599bc700ff2d33230711fcdc459758451
+size 434759225
diff --git a/model-00003.safetensors b/model-00003.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..74c7426b145cdfe204c7acb296c773f328a8c96f
--- /dev/null
+++ b/model-00003.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:038ed2e8f41354037a5d295398f3364e63df1f9cd9d7b488b530bb0ed77d1fb7
+size 434759229
diff --git a/model-00004.safetensors b/model-00004.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c9affe66029495321cdd87b3de474472ea355e50
--- /dev/null
+++ b/model-00004.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b724d7ebb3f6b4e3b7498f8e9baacd5d2530087f8f1ef0046a540f81c7fcdbf
+size 434759239
diff --git a/model-00005.safetensors b/model-00005.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..874c2bde8733ab30acc96fdd7f6b71cf3c091c71
--- /dev/null
+++ b/model-00005.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95f126b962239e8be7ada015093ed2c00eebaa5ce39a70f0b1d8055284c01547
+size 10470680924
diff --git a/model-00006.safetensors b/model-00006.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1c16a25a9aeed649da218747361bb3af140ca6ba
--- /dev/null
+++ b/model-00006.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:484ee3f58ddf86d6bb0ae6bd97e3451ed0414898d6a6e24d6d09ac0b53c6812c
+size 15598275
diff --git a/model-00007.safetensors b/model-00007.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1b622eb52e1bf79fbebc7bec841ecd275b549751
--- /dev/null
+++ b/model-00007.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb5a1b487d070462ba53cbc0dd922a8521a48835fc3ea0c9f253700acedf29d1
+size 10470680926
diff --git a/model-00008.safetensors b/model-00008.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b1e32535ca30b8070c66f9f3ff0b01d9efa5b2bc
--- /dev/null
+++ b/model-00008.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a3444d8f4a50f9bc0437a10694ebf1681f0fe719900e5b275d405205c6e5e22
+size 15598281
diff --git a/model-00009.safetensors b/model-00009.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..01e0883549c9e84df071d1a136975eccf5a5d820
--- /dev/null
+++ b/model-00009.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58074190afd2352363b0293ca4edd442d3402f64afa2601ed6e3e4094a1bee36
+size 10470680932
diff --git a/model-00010.safetensors b/model-00010.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b6f187ac1e34fcd9d5c7419344a2c2ef7d858190
--- /dev/null
+++ b/model-00010.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3070bb27422f350aaecd126a5e7ca7f23a2a0b838f366a47dcf38b1cbaed89e9
+size 15598283
diff --git a/model-00011.safetensors b/model-00011.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3a596d7e70b20d5610080cedc6317531965c4deb
--- /dev/null
+++ b/model-00011.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1aeed56502d16b834222f1a8132902816d8f2520c3f1f6c1952abf30cdc7f70
+size 10489425350
diff --git a/model-00012.safetensors b/model-00012.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..89b99e3eeb7e3b3a5959c29013d2245105c0f99e
--- /dev/null
+++ b/model-00012.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee1453d84a564349a41b147b8d43ce68cb00269694229506c327cce2d98d5b5d
+size 15598283
diff --git a/model-00013.safetensors b/model-00013.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..542fb0f2c9b6acf0130925115562454eea8b74cd
--- /dev/null
+++ b/model-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fd43bb75856f80a43d418ae41833ef797983214d96983cfd4427619ea58095f
+size 10470680976
diff --git a/model-00014.safetensors b/model-00014.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..cac6ceeacb05227101a8a1a36700045a52ca0289
--- /dev/null
+++ b/model-00014.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c083bfc09358f54fbf3af345557ebe2a4a1f5111ab7f1fd408040ec6b36fe877
+size 15598283
diff --git a/model-00015.safetensors b/model-00015.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ffd0d3e848b7142784c68fd119a6c10f6178390c
--- /dev/null
+++ b/model-00015.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdee21a18bd69255a8e260464d7439c8071af204cdc75d2ed07733881aa81f4a
+size 10470680902
diff --git a/model-00016.safetensors b/model-00016.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..f2d12363c738b160d24eb0ef0dac31a591d0f1f0
--- /dev/null
+++ b/model-00016.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c96f10c289b611ea60eb332fc44c5f329887116c23ca0c508531872d0edabbfa
+size 15598279
diff --git a/model-00017.safetensors b/model-00017.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..53c0fe82f7dcc4cf808ab62ff8250958e1fe780f
--- /dev/null
+++ b/model-00017.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056e34cbb02eacee71e6986f07acef096c245014ffef08d13838664cbc8cbfae
+size 10470680946
diff --git a/model-00018.safetensors b/model-00018.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e43f6d5031006cfb54f5d9247f5e12c367512a5b
--- /dev/null
+++ b/model-00018.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b6086984fdbb0447145d215003e7f2483f50a3db3712da31e9c342445b241c5
+size 15598279
diff --git a/model-00019.safetensors b/model-00019.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2a029329474ba3c3f700d18d7407ae9fe7fe883c
--- /dev/null
+++ b/model-00019.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81411890ac0ab9bcf7441efc896013bde2c7a9b1d7e8b76aa577a0556c69bd01
+size 10489425379
diff --git a/model-00020.safetensors b/model-00020.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e1903e0304a411c81b80ab6e1edcbc585f87cedc
--- /dev/null
+++ b/model-00020.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b246afcf8dfbb0a6abb7566afdab74308f8721810021002c4e94521a8e5d1d73
+size 15598287
diff --git a/model-00021.safetensors b/model-00021.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b367468079b26668a69aa8602b323d2966f1aedd
--- /dev/null
+++ b/model-00021.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79c5cb4a35d814adf299b85f25925dd533448f348d06c1d7af4f3f2aa73747f8
+size 10470681002
diff --git a/model-00022.safetensors b/model-00022.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b65890b616dda5bcdc51a33f40451f610b1d4c47
--- /dev/null
+++ b/model-00022.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55ff58a58d2c3804cdbc8f460344fe709b9e80bdd82a069347a299aa5f96790e
+size 15598287
diff --git a/model-00023.safetensors b/model-00023.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9dd0ec88d24e91af9b6699276b743c7ad09c41e0
--- /dev/null
+++ b/model-00023.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8005b6f5e867ae6c226ee8d658f960001884028b267fbf330f1e94cbcbfd7372
+size 10470680932
diff --git a/model-00024.safetensors b/model-00024.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0392c228a9bb10e9e27051ee9b37028944f5a201
--- /dev/null
+++ b/model-00024.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:997475dda5d4c7fa3f3bf7de6fa38d2a03aa491b54e02eedd2b91a4620d9001a
+size 15598293
diff --git a/model-00025.safetensors b/model-00025.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7c45468af36393d14f6ac6ca08da992169df2da3
--- /dev/null
+++ b/model-00025.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59c489271d97af83dbd67e04d3470e57c923faef57a52a102f91d46e64dc2794
+size 10470680962
diff --git a/model-00026.safetensors b/model-00026.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5575c4d8757e871049a75111b679480622d2701f
--- /dev/null
+++ b/model-00026.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:898623b045c7bed11180f439c4882691066963104df2c8c7b132a6b5464ea2dc
+size 15598293
diff --git a/model-00027.safetensors b/model-00027.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8d5ad15285b421a8128ddfd0f8817d7c01a3f83c
--- /dev/null
+++ b/model-00027.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d43ca9298b6df20682ea8ea2509c2a5c9cd365270d21f9f3944fc4d2bb7fae8
+size 10489425419
diff --git a/model-00028.safetensors b/model-00028.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..20b3306d6376e80d3ed7e3aec1c1f24dd4524e2d
--- /dev/null
+++ b/model-00028.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5136d7e36985a70b2bea8219f314524cb79cc85845dec13f57454f38eef0a482
+size 15598289
diff --git a/model-00029.safetensors b/model-00029.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2be21e3e9a5835247cb6d8e141bde29ba67f9cd5
--- /dev/null
+++ b/model-00029.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5f4e0f2ca26d2e3d69db6167ed111d7edda955d078cfb38dfd13e97ffdcc2e7
+size 10470680980
diff --git a/model-00030.safetensors b/model-00030.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b23f855dd491c0c96c13c2f7ac00900a8ca8ea4c
--- /dev/null
+++ b/model-00030.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:962eeecc6fad62b26f639a5d0d33411097b33df797a64062c1283a4850855745
+size 15598291
diff --git a/model-00031.safetensors b/model-00031.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a81c24c9aa1e67e109101d3b093399259d57a94a
--- /dev/null
+++ b/model-00031.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73775b9b679de291946cc5f8ba4cd9ac5e7c721475f712445a3fcd137b7ea566
+size 10470681012
diff --git a/model-00032.safetensors b/model-00032.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..49cec40a4e7e8899ab680f906a6b0de1ccd871c0
--- /dev/null
+++ b/model-00032.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c00263bf4a60a7cbea87b7b958fa21d2246f86ef15a1362588934bf45f4feefc
+size 15598285
diff --git a/model-00033.safetensors b/model-00033.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c0d093f4b787fd6adf9d6dd10f0b44da44757a21
--- /dev/null
+++ b/model-00033.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5ff09fb304627c647cf8251e51e2b07e007d78df56fda59b976f207a6f517a9
+size 10470680952
diff --git a/model-00034.safetensors b/model-00034.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b97050b68b908d92c42fecab577dfb75d4cf23d8
--- /dev/null
+++ b/model-00034.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5686f391d4ab32b60585aedd907e9ae6b1ecd20e11cbd13ec710de1a45d6190
+size 15598281
diff --git a/model-00035.safetensors b/model-00035.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..dfc948707bfca82e5f7f31383ce4ac39c9ab7dde
--- /dev/null
+++ b/model-00035.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fa015719d715d25a352507f3f715299caf231a1fa26eccb810a344ed33bb1d0
+size 10489425433
diff --git a/model-00036.safetensors b/model-00036.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2a2ab83be7134c8d6fda2512829a64d4f5e75e93
--- /dev/null
+++ b/model-00036.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd3cc65b941b18cab38d55e428e5b799535740c6467cd42e243f00c7b686485d
+size 15598289
diff --git a/model-00037.safetensors b/model-00037.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c2cb2967c91e2f4c71e982e53e911bc6d0f6faf5
--- /dev/null
+++ b/model-00037.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ba1b5751c29f44dff69999f4aeebdfa98d79334a9a8dca05eb92a53f942631e
+size 10470680954
diff --git a/model-00038.safetensors b/model-00038.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..61a3971ba7b9e3a3cd1a087f73e9898a4ef2b5b8
--- /dev/null
+++ b/model-00038.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ad2b7feda0bb4b14cfd865ecfb1b8b471133c238e0fae669f9d81dfd75ef614
+size 15598283
diff --git a/model-00039.safetensors b/model-00039.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1b456a14c58f6ae1ee0f1fb929b2bcda025786ea
--- /dev/null
+++ b/model-00039.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae052d4c2ed560737d5f90ed7a3a47e3f3170df868d5e6ed5ad3d934284a7f6c
+size 10470680986
diff --git a/model-00040.safetensors b/model-00040.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6863803efe42e4c05a876497bc9b623009a16ae1
--- /dev/null
+++ b/model-00040.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea501fe95eb3d8330ab4409e141738280aa2c79df5dca525a2be0f5da5447634
+size 15598289
diff --git a/model-00041.safetensors b/model-00041.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b07d1d09a367f5b5c2ceea49fcc4dead2925af6e
--- /dev/null
+++ b/model-00041.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07dd50b871c0506ac3a7c6b654930db8efac239c5b734f11614237ed9b016cd0
+size 10470680996
diff --git a/model-00042.safetensors b/model-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0e83e4937d22b9d713ca5d78858db52864e725e1
--- /dev/null
+++ b/model-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:355b70f1f1e08f2148a97907ddf4cc663cef5a51572a30617c0c4d356f0ea178
+size 15598281
diff --git a/model-00043.safetensors b/model-00043.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..23dce61bd275453ab50891aa8a594b446f7b2780
--- /dev/null
+++ b/model-00043.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfba091ba5ed320d9f4252a5819ab100604acb3b34db7f0ed6edf8f658a9a0f3
+size 10489425431
diff --git a/model-00044.safetensors b/model-00044.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bb5b0289225619941ea514360841810a9d9f56d4
--- /dev/null
+++ b/model-00044.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc5a81af462eb678f7e694e8e2a56c0fea25324cb781375c9c973b12b053ba51
+size 15598287
diff --git a/model-00045.safetensors b/model-00045.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..56a6d912dc6620f9ce4940c6d37fb324d89777b8
--- /dev/null
+++ b/model-00045.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80414ea559cc277c47dee1b07b25649b9e7beb21766658e058f0841c57045446
+size 10470680960
diff --git a/model-00046.safetensors b/model-00046.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c9d07e1caf39773910593efb5b5ead07e87be0e5
--- /dev/null
+++ b/model-00046.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:291ac533eb33d6e02448e60916fff0d5f2e16ab4c60e38f99f65629edafd2db7
+size 15598283
diff --git a/model-00047.safetensors b/model-00047.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b84ab2af07ac334324a7ab02348f7b66d9dda38c
--- /dev/null
+++ b/model-00047.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65b5bc1a510a5676cc0d06eda95628e5a66f9d36b5719e4d572dbfbf4a2d7169
+size 10470680988
diff --git a/model-00048.safetensors b/model-00048.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0afac8742219cb3eed16555034dbf0e1e082933f
--- /dev/null
+++ b/model-00048.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54528733eb61105da2f46bb8c5d303527002ed1b04716fe4e031e56ec2985a87
+size 15598283
diff --git a/model-00049.safetensors b/model-00049.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..95acec575126832f296dd291dd26bf099eeab5bd
--- /dev/null
+++ b/model-00049.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d635c3249f0b2012a505fa11d15f77f0b2b6ebfcf0142dd30c130c4e154f2f0
+size 10470681012
diff --git a/model-00050.safetensors b/model-00050.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2857bd69a4901111bb202ca2fd22113317476e58
--- /dev/null
+++ b/model-00050.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c5151a87fd69cfc19b28c32cba4c1ea839467837b2a0509a7b14ba3789e26b0
+size 15598285
diff --git a/model-00051.safetensors b/model-00051.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..fbda4aa9e3156917bd57b3cedc96dc7175a27e5c
--- /dev/null
+++ b/model-00051.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a2942ec8066c068351851b0dc9014ceabe4e7f780820d6c6a956c17e2f7d89c
+size 10489425405
diff --git a/model-00052.safetensors b/model-00052.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c190187cf101762c072eab577393d2f9f9ab0da0
--- /dev/null
+++ b/model-00052.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ac697cb2f12ff594fea8ec485944fc064a97c46c4c4403d0f5d914beb4d0a4c
+size 15598285
diff --git a/model-00053.safetensors b/model-00053.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8ec27793583d2f2961f7b1ca844d0809880392aa
--- /dev/null
+++ b/model-00053.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:551c64bb027dd694b4ffd680b564b78bda9d33151f12891652223b122eb039e5
+size 10470681010
diff --git a/model-00054.safetensors b/model-00054.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..339642c40b64d49e55912ac0190fb26c589ff221
--- /dev/null
+++ b/model-00054.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ec036fc8059112913b5275a8122a15a25ef6d1ed72671a1c3ac970c9d259561
+size 15598283
diff --git a/model-00055.safetensors b/model-00055.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..06e4c183afde8b520b07095b77ff1a0ebd90452a
--- /dev/null
+++ b/model-00055.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5399d16a9d7841d71ef4291d55f856b22e42189cf5b1af25be49b2d65e32083b
+size 10470680970
diff --git a/model-00056.safetensors b/model-00056.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3fde92ca07dcccd00f0282d28cc09f3878645030
--- /dev/null
+++ b/model-00056.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6c6d85b0b89ac87665a963b18e0ca387eb5f77e3898a7575197c975962a66a2
+size 15598281
diff --git a/model-00057.safetensors b/model-00057.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..912bd2e26850555ba89c6708d147ed307a6ede9c
--- /dev/null
+++ b/model-00057.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdd60078bb197b47e736efa8ba3e74153d8c0a4cb820d3cc516727b00a502725
+size 10470680962
diff --git a/model-00058.safetensors b/model-00058.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7787134be4181f1a0c1bf6af13162359469b9f81
--- /dev/null
+++ b/model-00058.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc1af32d52925824277c035e763262c5a36ea1d4ec5afac5e88ab42468084cf1
+size 15598283
diff --git a/model-00059.safetensors b/model-00059.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ed9af8b8c7fdaa71a48a5b81a10ebc77b3b57f77
--- /dev/null
+++ b/model-00059.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23179edc632afa3f7a32b53d89c9fd191cce2cfebc3c7c09e9c5449b5db595a9
+size 10489425381
diff --git a/model-00060.safetensors b/model-00060.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8e0dc999ac6c993d55ce45ed919b2579b214c285
--- /dev/null
+++ b/model-00060.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66936ebb150c9d8819a7bfb6d18f19a344d30cb6246f6dcc44d61e66e4e43af3
+size 15598289
diff --git a/model-00061.safetensors b/model-00061.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..aa9261edc4b7e1c61b1996981615723ac842c8de
--- /dev/null
+++ b/model-00061.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59c21d1fabde65fa73104a8a4c3e4f1030234c0f5010202834aaf4cb343f45b3
+size 10470680992
diff --git a/model-00062.safetensors b/model-00062.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..39924257add1888fe40de68702cff64f4765ae20
--- /dev/null
+++ b/model-00062.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fe2ed332d1c6ac258d67b39985dbe68909f28611deeb30d48ac32096868a165
+size 15598283
diff --git a/model-00063.safetensors b/model-00063.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..dc58fe4ea20244c4d29cf8b223a7003cbe5d81e4
--- /dev/null
+++ b/model-00063.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0da5c8d2122dab8156039021f27b297d1a24ff9858f819e672255f824954ab75
+size 10470681008
diff --git a/model-00064.safetensors b/model-00064.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3c2e04883ba5643c32d0043859b97e5adacec156
--- /dev/null
+++ b/model-00064.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:387c9093223d48ac69efc5b955b8bdaf791b4f2e651512ce75bba02af20ba46f
+size 15598285
diff --git a/model-00065.safetensors b/model-00065.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bc1f94087f2d1050fcfba0e86c4133a917cd0b85
--- /dev/null
+++ b/model-00065.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc3aa974b86940582f97fc2b74fc0b522aa05bb28fd8a1ff07460b0686594c9a
+size 10470681010
diff --git a/model-00066.safetensors b/model-00066.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b4a1d443f9eaf1f62475600e60dee48b88539fda
--- /dev/null
+++ b/model-00066.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c05c5bc8233d662bd3a944bc5993c801edfeffc5f6e4cb091ae72a2c97e133d7
+size 15598289
diff --git a/model-00067.safetensors b/model-00067.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..88a81e250b234d869cda8d86c727e681a101bcbe
--- /dev/null
+++ b/model-00067.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5d03f0178d52d41e1ff96a56c218d6e87e22a595f9cb8be08014fbd1e21997c
+size 10489425365
diff --git a/model-00068.safetensors b/model-00068.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5fb7e0311d67ad48aaea2fc82eef1e8adbbd612b
--- /dev/null
+++ b/model-00068.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b60b3c59c88b28fd82f4f1d6316d4615ee24f18a4071ef3ba466152d9a7e616c
+size 15598285
diff --git a/model-00069.safetensors b/model-00069.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e324e30fb3e668880ddf211d7e5b6014f0a0cb57
--- /dev/null
+++ b/model-00069.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f873dfbb0cf16b702e855576b0dda10d161ab3e39457d17c0b55d84904aec35
+size 10470680996
diff --git a/model-00070.safetensors b/model-00070.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b31c6dea058634722f28ede8497f370a798fc3aa
--- /dev/null
+++ b/model-00070.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02085fe858edba2e90963e6af2512a524507de067788ab04a1cdaf9c8012d75c
+size 15598283
diff --git a/model-00071.safetensors b/model-00071.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ef86bfb8362ef4ba5d4b8e690d72b102b67217a7
--- /dev/null
+++ b/model-00071.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14e543d1a4ea2b08e2de227053679e74e9603ff9f39e22b75f318b0c7cf5b6d6
+size 10470681010
diff --git a/model-00072.safetensors b/model-00072.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3e6599a0f10a290329ca0ba2b9d3cd94bd8c2ba5
--- /dev/null
+++ b/model-00072.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f887b71d3e36fc1e1d5120b60467167f0c5a408877a362ac593310da0e608f8
+size 15598285
diff --git a/model-00073.safetensors b/model-00073.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e1c03b4db20b780a86e9cec9de20a4de0caf1def
--- /dev/null
+++ b/model-00073.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f808fd8afe92cd7c9cd7c466c37c583d2927f4b4b5dc544dbc1e2be85296c209
+size 10470680970
diff --git a/model-00074.safetensors b/model-00074.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ca4ce40764e63f40e699fd46a365e37c211d0bc3
--- /dev/null
+++ b/model-00074.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c50596742a09df01207a3861b00642430785845088a410cf1646ca1bce6c35a8
+size 15598285
diff --git a/model-00075.safetensors b/model-00075.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8f42af2a6a117a18a4691f87288272acd85be82f
--- /dev/null
+++ b/model-00075.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4bf00db79c06ea8a7417b1c92dc0c84f82d216b268d77c7503515a92a3f0317
+size 10489425381
diff --git a/model-00076.safetensors b/model-00076.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..31e15cec88902a12b5912129cd29d1eae1858482
--- /dev/null
+++ b/model-00076.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d24ed3c9a36b2fe394dd5dc20c3cbeb4b7a856b8df12149aeb1ff58b37baee85
+size 15598285
diff --git a/model-00077.safetensors b/model-00077.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d0ad31e38591f927bb4c8e2e9ea979dd2620469d
--- /dev/null
+++ b/model-00077.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f448770bbfe4478defe58bda2f50f3fb9e9d98df823e6b880e2b690fd8032d23
+size 10470680956
diff --git a/model-00078.safetensors b/model-00078.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c65532d75c537979bb7f4613a7159feba18452b5
--- /dev/null
+++ b/model-00078.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cfe5f3841146315de0ae8f5a74f4ac1d13168975e3a7a81ba487181841ffac8
+size 15598287
diff --git a/model-00079.safetensors b/model-00079.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9fdd79f94ef5627c78043faf6c02aca093ca26a9
--- /dev/null
+++ b/model-00079.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6d735082d8937c2e5107de72ec3f43b4fb41148ab45e26211a7c545c8a6c79c
+size 10470680966
diff --git a/model-00080.safetensors b/model-00080.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3f1d9ad0a9597f7cbbc4fca6da6ebd24acba5210
--- /dev/null
+++ b/model-00080.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7240144c529e20415b09e370b7fbfcff3965933907792ff0916b78bd1ae405f4
+size 15598285
diff --git a/model-00081.safetensors b/model-00081.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bdfaef4d0ba8e42395f3404d051af23beb6d62a7
--- /dev/null
+++ b/model-00081.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d289eb59b8269b7f2f8acb25a25d0d65be0c491e29296ac55f4a890e6b451957
+size 10470680994
diff --git a/model-00082.safetensors b/model-00082.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..17338750156f38a0cf6c168d06d81302f8b5d5e7
--- /dev/null
+++ b/model-00082.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0bdfae1480f8d60d47d2f68d27a7bbd3e05b725d898117585974610cb7ee24f
+size 15598287
diff --git a/model-00083.safetensors b/model-00083.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a1a418be0c2220bef5d7ad109b779eb5b11ef54c
--- /dev/null
+++ b/model-00083.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5d4962c50100cdef36e2e0319c81071112895736dcf77504604fac861d96166
+size 10489425407
diff --git a/model-00084.safetensors b/model-00084.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b5fd9ebefabed81a5ce16d31ac80efcf7954647c
--- /dev/null
+++ b/model-00084.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:033c3287a229f8ab8200c6892d2b195bb4ec4688d92910c231abab363c01df26
+size 15598285
diff --git a/model-00085.safetensors b/model-00085.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..425ce4c714fd0f8ac130ba142fc5cb2e29d17740
--- /dev/null
+++ b/model-00085.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4afaa0e34093dc552f7ff48a0b75b0e575ce70648eb909c0e05ecb77ef8b7b24
+size 10470681000
diff --git a/model-00086.safetensors b/model-00086.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bca74db067db7651fc47bd997a63d96d97d92279
--- /dev/null
+++ b/model-00086.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:347d5b0155085496a1bae57b237eda244b9f9af5f162360ee56db4fd8e803dfe
+size 15598287
diff --git a/model-00087.safetensors b/model-00087.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..88fb27c6144cd2010548ebcf9ef639234cf256d0
--- /dev/null
+++ b/model-00087.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecc9d48dbfb55776d9cc705f5d93af97e30067804b704fc9b5429a81bb142fa3
+size 10470680966
diff --git a/model-00088.safetensors b/model-00088.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b99816f93b133c186c7234d883d0e3c032096e6a
--- /dev/null
+++ b/model-00088.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c3f0b0822cb8f89b9cb7267b5e35d65e7858ca6bbc231e1d3b3d6ec655acc23
+size 15598283
diff --git a/model-00089.safetensors b/model-00089.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8087598795cf0ab0ace75cc3b3eda618118e2941
--- /dev/null
+++ b/model-00089.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84cd1cab18a00793c97bfaa62c6e4aafc80e9747b609aac846170b9b3121eeff
+size 10470680966
diff --git a/model-00090.safetensors b/model-00090.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b29207a681b7d5750f7d56dd9eabf1440c9dc71c
--- /dev/null
+++ b/model-00090.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a81869994c69f81c6a6a9dcc9048626fb77de0cdd7db3c122380d0b4574d3dd6
+size 15598285
diff --git a/model-00091.safetensors b/model-00091.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3977484227c89378daa6f1a757cc8585eb1d149a
--- /dev/null
+++ b/model-00091.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99f736eaf6c1cb134424ccb554fe4a1845bc098179f61e232acb7f4f17743e3a
+size 10489425431
diff --git a/model-00092.safetensors b/model-00092.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c2726ca392589319d9dcde728aca87b57ba163b4
--- /dev/null
+++ b/model-00092.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2806ced2b7bc6173ad873a2f6bea0444ab86e2abd710651f033a8b7299b727b9
+size 15598285
diff --git a/model-00093.safetensors b/model-00093.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4160bec7ba6aa9ca7bec4f9895684edbd5c1acb
--- /dev/null
+++ b/model-00093.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87d3015e4868e080f68dd8ab2e6379edc3a628043536ea5b1fd7bbffdb53b9dd
+size 10470680976
diff --git a/model-00094.safetensors b/model-00094.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..39f32088c0ac7cb108fa1de49e0596c8cccedae0
--- /dev/null
+++ b/model-00094.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83e40a5b82d22d03111c0ae2bbdd6878410480c0b65aa25911fef6edf3ea5b90
+size 15598283
diff --git a/model-00095.safetensors b/model-00095.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bbe0e5a2d5b0b3f83bdcbc1fbfa6459aa87f68d9
--- /dev/null
+++ b/model-00095.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a07cd76257215280599c1cbf47b6f984f4c367c755a77f6f932696084d3a4d29
+size 10470680962
diff --git a/model-00096.safetensors b/model-00096.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ac173864cd4d9f0168af1cec6465caa862f6634b
--- /dev/null
+++ b/model-00096.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41a4151e958bb20621e7676883cfc2ff0139ef1eb451b892c0391fa7e062dbdb
+size 15598287
diff --git a/model-00097.safetensors b/model-00097.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d4603314a36488b1bf78209f12f8032ff89c9f6f
--- /dev/null
+++ b/model-00097.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ac3211ec64eac143e3f75249a720793e3f574f02970ee495368982f84f06a74
+size 10470680966
diff --git a/model-00098.safetensors b/model-00098.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ddfa4f7e74ce54a65f166552a1534db6264fb324
--- /dev/null
+++ b/model-00098.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:686b0f9fbaa32b8b4e7e1fb5247ec18d74f4dbe097f0446d9693f5d6eb2f574d
+size 15598285
diff --git a/model-00099.safetensors b/model-00099.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..362052456fdd329ff9c421ddb41aeb93ac7296c0
--- /dev/null
+++ b/model-00099.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56bbfae710be56e66a75aa3455dfd4962cbdf9cc7656f63785d9374c3087280e
+size 10489425381
diff --git a/model-00100.safetensors b/model-00100.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d5cb52f775d4423e4b6c5cba9bfaaf916da9ad1c
--- /dev/null
+++ b/model-00100.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74e531e458148f134c24fb5b2e926a482808eff340836fb98dfbd4606575bb90
+size 15598285
diff --git a/model-00101.safetensors b/model-00101.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..38c03cf757657facfeb72bd54208ec217d5b3a9b
--- /dev/null
+++ b/model-00101.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35ed4315bd4d4f26554d7b0601296feda2685ccf61b80aabbd672747c06f9ea8
+size 10470680992
diff --git a/model-00102.safetensors b/model-00102.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..94b64a87cf3448f503ca26959f6210f7b6593e2e
--- /dev/null
+++ b/model-00102.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d7573917eee61cf47863ef638b9da776ae936877d32f611ef001a105bb6efbe
+size 15598283
diff --git a/model-00103.safetensors b/model-00103.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5ae678473844b61e1c8580b05e40f8c651eaf3a1
--- /dev/null
+++ b/model-00103.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28255b00025c6fb6a515a93a4bdc7ba4aba6489b2b39755c9096fde04ff625b6
+size 10470681000
diff --git a/model-00104.safetensors b/model-00104.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..68e68f964e888c7b46e6807cd2ae194981c921e0
--- /dev/null
+++ b/model-00104.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a841ab1f047ba08084be9083fb6685cb92c84c8f39244b8493e2684506e504d1
+size 15598293
diff --git a/model-00105.safetensors b/model-00105.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..873fd550b51f9b681e3c1e0fc4f3367f78eaba95
--- /dev/null
+++ b/model-00105.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebc0286efc364e1233ba4d6e6bf08cd7e98787d6f170a92e70017d10d186c12e
+size 10470680964
diff --git a/model-00106.safetensors b/model-00106.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0eaba809c69cafb81e49ca5cb1840f66c6535f74
--- /dev/null
+++ b/model-00106.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b048f3ab1ebd49a2060d2788d8a48a2185bb472844e65f9ef92e158090c12cfa
+size 15598289
diff --git a/model-00107.safetensors b/model-00107.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..261aff776f23b884e6b5fb3ca223d9d253cb558f
--- /dev/null
+++ b/model-00107.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6eb3f021346f25464b09a67efdb8aa58faf1ffc57aee12ae9124a6efaed1b9d5
+size 10489425415
diff --git a/model-00108.safetensors b/model-00108.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5fce74bd457b9313947f1e8a4428ba27cabc3d00
--- /dev/null
+++ b/model-00108.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb60414ddcb38c2c4e20f45088d86a0b74c1e218c821e331d940a66cf941d396
+size 15598287
diff --git a/model-00109.safetensors b/model-00109.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..15dd0a69656458eef106d60154a41fa723a1fabd
--- /dev/null
+++ b/model-00109.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f88e54c6f01f79878185466a90217b7687fde209a30f2dedd6c2133c441a07a0
+size 10470680970
diff --git a/model-00110.safetensors b/model-00110.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c8d9df78f4b53a4527eb12d8c35147a996b03a89
--- /dev/null
+++ b/model-00110.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:beba98e5147304831a9d0d9f9be9215a3d0554aa29309541ffe1d36dfffc3dc5
+size 15598287
diff --git a/model-00111.safetensors b/model-00111.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6154f3c1c7c96fa6f34772f84144557eb06722df
--- /dev/null
+++ b/model-00111.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43d00a7a803e8882c63a32abf7d618f7081d6cdaa437249898820243fa3759c6
+size 10470680996
diff --git a/model-00112.safetensors b/model-00112.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ba45ecbe3ef4bd2ef3b29c4888783301ccd7636a
--- /dev/null
+++ b/model-00112.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df07a794981ab9e24772e932fde89b830cc045d108afbffde21d5841795e42a1
+size 15598285
diff --git a/model-00113.safetensors b/model-00113.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9e35de8fc1272e4b9300c6f3157f7750d244129c
--- /dev/null
+++ b/model-00113.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81aef0dc1b680af6c749870b73afc3bee80d42e13aca7f7692e35b01f5af316e
+size 10470681012
diff --git a/model-00114.safetensors b/model-00114.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..f8ff035db90d159f8f948ee9cc4c0ccdb7ba72a8
--- /dev/null
+++ b/model-00114.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c21a7d4d94f8ac1f4c368b14969b8cd7ee4c2acde486b26267a41d71d1538508
+size 15598289
diff --git a/model-00115.safetensors b/model-00115.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1e272f3c54ca7e24c319918517130ec18df95748
--- /dev/null
+++ b/model-00115.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20e28f2b4cee9f33cd93f0d9f423ae0cc2e333dadf0e2ef4cb97f115fd5a5e41
+size 10489425377
diff --git a/model-00116.safetensors b/model-00116.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4ab516b8e90f9f08c0390b1cc96a1b693c07e5e
--- /dev/null
+++ b/model-00116.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f596d7c7d2f04a03b6a1c2ae03c0bc9f49dad293e0902df0572868e182331b2
+size 15598283
diff --git a/model-00117.safetensors b/model-00117.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a4465878942f21036991c08d1fbb8a392382e433
--- /dev/null
+++ b/model-00117.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3452339422ca4bf3340670fc4057a8d509fcaa91b30a18bc7039eeb445dc3e06
+size 10470680976
diff --git a/model-00118.safetensors b/model-00118.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..99be06b045f3d014306c2e14e7b5134bc77d80a3
--- /dev/null
+++ b/model-00118.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ab158364d5f5ce50fbba704e0d08cf9a44ca9279a5d1e37308124fdf375599e
+size 15598287
diff --git a/model-00119.safetensors b/model-00119.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..136a76313f4697a6dd02e6bd716227110f0f1ae9
--- /dev/null
+++ b/model-00119.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cdfcca5e8e1d742db69eb644be5bd0e4bcbc0aa0dbb47515eb893a868e381d3
+size 10470680948
diff --git a/model-00120.safetensors b/model-00120.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..79fa43e509288bad171d4ef48e4f87f1b923f291
--- /dev/null
+++ b/model-00120.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d8be72fc989a358342735582d7698411c4ebc1a7a693855972486b794c37a21
+size 15598287
diff --git a/model-00121.safetensors b/model-00121.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2da27b7ceab5fa3643a87a9cdf7cacacb33e2880
--- /dev/null
+++ b/model-00121.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c45b2b0f8f067581b13c1738f2a88e956a4e2d771bab8d96cb74a6976a3e5bc0
+size 10470681006
diff --git a/model-00122.safetensors b/model-00122.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1419321b80b9b3e7057908629aa5ab1eb3bba3e8
--- /dev/null
+++ b/model-00122.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3e0b78715edea39caae4f0a9ab465fed7a6ff1425d703d963372795e55a9c77
+size 15598285
diff --git a/model-00123.safetensors b/model-00123.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..37a87f5b8796025e9dff5c92b1ac1de8670c45ce
--- /dev/null
+++ b/model-00123.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e16e3a5ac057cc0bf966981def6b6b62a122ae6be406ec24a19bac3ab3f993f1
+size 10489425383
diff --git a/model-00124.safetensors b/model-00124.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..03ad770c6cda6e6956ded3e2ad879eacdcdbffe3
--- /dev/null
+++ b/model-00124.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b739d33cdc8c62af6a0b90f6726526425c93c783f24339e41be63b805740a3a
+size 15598285
diff --git a/model-00125.safetensors b/model-00125.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a83432e0b71e81dbe05e89dc1b5f092c267467eb
--- /dev/null
+++ b/model-00125.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c7019de1eaf2244b9e82e27f87f360d482e62acba46076eabd5e3274ae81f5a
+size 10470680956
diff --git a/model-00126.safetensors b/model-00126.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e29fa5097f861e4f574505bd21583ce703167128
--- /dev/null
+++ b/model-00126.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c57f2f87babd8470e4004ff918ea3e37281e7b24aa88d07d9fa1ab5d44ebe8ac
+size 15598293
diff --git a/model-00127.safetensors b/model-00127.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..73b2f2866eca5935f5fa70b61e58f113ed21c293
--- /dev/null
+++ b/model-00127.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61da34adc0b29ea3bdb6a8ac3ad4b132f189909b7786c9a0995f47030f7101f4
+size 10470680960
diff --git a/model-00128.safetensors b/model-00128.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3b060f387cf0f027cb12910e3c95d1b3613434d6
--- /dev/null
+++ b/model-00128.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21787a86fe0569b3f21d11f7781267e28a27beb73d0571d520fcf248db845e0c
+size 15598283
diff --git a/model-00129.safetensors b/model-00129.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c68ba9180f29995d08178b6e17930007d1abdedc
--- /dev/null
+++ b/model-00129.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de3e4f3dfdcc5cbf562ebb8e0bcbf0cb0f7f1721ace1d3d6f374df4708915f42
+size 10470680974
diff --git a/model-00130.safetensors b/model-00130.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..eaa3c59653ac4bd72bd918f985b5c224ece11e80
--- /dev/null
+++ b/model-00130.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f6426c0cf9de8506630a49df06861a49a4779058c0ae33b1337645e36b09238
+size 15598285
diff --git a/model-00131.safetensors b/model-00131.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..93936818f902638774fa8b21ac7de9991b2a8e45
--- /dev/null
+++ b/model-00131.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd4792f36102c842b8593def06fdd430371562753084fe89303945866dc2ac13
+size 10489425393
diff --git a/model-00132.safetensors b/model-00132.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..61b3fe2e370ab8bba4cb4dd2db0a66e875fb9713
--- /dev/null
+++ b/model-00132.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ff7e50d7d3bf35ada9e5483e4d0a06eef9b49c8c2248f10b7fab82a309ccb92
+size 15598287
diff --git a/model-00133.safetensors b/model-00133.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..22d8476547d32b1172ef945a0459ede36dc24ae7
--- /dev/null
+++ b/model-00133.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c72879fd200e542123952e04091f39d316a9b2eaec3cd2111cc63c7895325912
+size 10470680992
diff --git a/model-00134.safetensors b/model-00134.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d8dcd469d3476c686e56e7f02d265945e47abc57
--- /dev/null
+++ b/model-00134.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9772d8c20640aac42a816daf8191e0cd2010eb675f4ef263d37646fcda4c0cff
+size 15598287
diff --git a/model-00135.safetensors b/model-00135.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..259eb18dd34232510e491351f7439cb1bf2caffc
--- /dev/null
+++ b/model-00135.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1cd458c4200dc682e6ffbc5e7ff506a00531ec2ca793ac5aba1ed04db3d2068
+size 10470680992
diff --git a/model-00136.safetensors b/model-00136.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..91b8e9e30aaca3facfea3f8bef66066bca256c12
--- /dev/null
+++ b/model-00136.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76fdf7d627f7e079d8ca712869419973497c08bd2c953612b7f33dc9a8d8b8d3
+size 15598283
diff --git a/model-00137.safetensors b/model-00137.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e9cd2b80332c26e272cfd6f5de652466c82dc215
--- /dev/null
+++ b/model-00137.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ff6fa4ba57266bce9f9a8a25d0ec98d24faa92f8dc420ad6a9e9245828199ec
+size 10470680998
diff --git a/model-00138.safetensors b/model-00138.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..84784830fe761e47ea9a677565507f3f907e9793
--- /dev/null
+++ b/model-00138.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e81604a6eef396e6660fa7ca8b38ea42e51b254b8585fd00e0e196826be4eb85
+size 15598285
diff --git a/model-00139.safetensors b/model-00139.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..fc88925274c4b01ed0f35ccaf97b79657be6aeb5
--- /dev/null
+++ b/model-00139.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:528bb692bef7b0d17ad91b1ad0eee9d8d5078840ea4b6269e8b8b1d7413cdea4
+size 10489425389
diff --git a/model-00140.safetensors b/model-00140.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..031c4d9c4069f26dec9d436a84102d12082b5f7c
--- /dev/null
+++ b/model-00140.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d45ee8486ddf59d43235ff3763fb5fb3bdfea8700181c906fe48f7af0e892630
+size 15598293
diff --git a/model-00141.safetensors b/model-00141.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..dc55b2f78bae96125dc820f92831d90738804da9
--- /dev/null
+++ b/model-00141.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2544f105bd2f1518f45cd4bd957c8346a173c48d151a74048d42a41c1d51e3e5
+size 10470680978
diff --git a/model-00142.safetensors b/model-00142.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..743f71d94f8b4ddc008d47f571e310b1bfde9f9b
--- /dev/null
+++ b/model-00142.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b01cebe36f0cc9c3ba28c5dd8d3dedbb32ec01100635f70fc49ecf00482a9845
+size 15598281
diff --git a/model-00143.safetensors b/model-00143.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..51bc2cf1310d9ca32e7652d40a6ec5975453cf0b
--- /dev/null
+++ b/model-00143.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfd83ccde1879df82b5bd7cb13ce776da9d31ced40b5bd04817f0508f5868d8a
+size 10470680970
diff --git a/model-00144.safetensors b/model-00144.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..573d10250b1804dbf933fb074d9100c34ab3a180
--- /dev/null
+++ b/model-00144.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73fd3eea9c27ca01a598c9830ef08888678759bff8b7cf075fb73fdd535468e7
+size 15598291
diff --git a/model-00145.safetensors b/model-00145.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..4020e7e92bd2ab549f37f519db322f932ac02661
--- /dev/null
+++ b/model-00145.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:380c0fbdd8322279829226b1ab21127e7adfbbe61929dc7b7740c820768853da
+size 10470680956
diff --git a/model-00146.safetensors b/model-00146.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..aa0ae6f9ce18a0f3e8c32aa9895576f680b3e03b
--- /dev/null
+++ b/model-00146.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d77d90a2c1b856224626a282a6014fcc75bd5ad17cd8539e1d1197a16d6f255
+size 15598281
diff --git a/model-00147.safetensors b/model-00147.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..08f958117eb981b9e3f454f8c51ad7b83f11c846
--- /dev/null
+++ b/model-00147.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6655b70abfef283d5d1c4bf10161c60a9af7991276132dbe13f6b45ce9482ce
+size 10489425383
diff --git a/model-00148.safetensors b/model-00148.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2d44a67c371757398acc159d26ffa48e3195ec97
--- /dev/null
+++ b/model-00148.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60aa661f511c89211f71dd291a0561e144be18de40edd7720945981b8aa51746
+size 15598289
diff --git a/model-00149.safetensors b/model-00149.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a9fb77a990af9229f25fcfe58aeb313faf95bfd3
--- /dev/null
+++ b/model-00149.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70b0652a4f91cab0edc651ea6d1c74c088e29ed4114426202e5e2f1f8bbea045
+size 10470681008
diff --git a/model-00150.safetensors b/model-00150.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..92c57d504e30f294a7e69e2cca7813a4eee66b61
--- /dev/null
+++ b/model-00150.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:607be90198b6131c999f7a45765162d5452566200e9532e8d2d3c9a596be36ca
+size 15598285
diff --git a/model-00151.safetensors b/model-00151.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..400c6465fe05eb602eecc3a7bca8ed76db0d328c
--- /dev/null
+++ b/model-00151.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46d2b21eaf350a76bd0c4ad996d54df3c2c1af0ea2e897a312f7c84cbf30e6ff
+size 10470680970
diff --git a/model-00152.safetensors b/model-00152.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..c6bf92c76d91c572da213e0f45f18914047778d7
--- /dev/null
+++ b/model-00152.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:926acecff484a06bb43c77f05dc6b3f61e4190860c7b375a0e9e5786e70a4ab3
+size 15598287
diff --git a/model-00153.safetensors b/model-00153.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6dd4bf071c46e3722fcffc2d00590f1f1d389182
--- /dev/null
+++ b/model-00153.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfccc8d78e71122c5d5417867d6707cc266c886894a622126c807b0cdec5a02a
+size 10470681010
diff --git a/model-00154.safetensors b/model-00154.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2a27eb64323a29c245ffa59fb1b67cf94638e546
--- /dev/null
+++ b/model-00154.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fe5137f591ce296e9f857026d07948beef6e33eaa09763f38a60bcc9761e3c0
+size 15598285
diff --git a/model.safetensors.index.json b/model.safetensors.index.json
new file mode 100644
index 0000000000000000000000000000000000000000..55eb167576d188daf01676df38ad875f524101af
--- /dev/null
+++ b/model.safetensors.index.json
@@ -0,0 +1,3362 @@
+{
+  "metadata": {
+    "total_size": 790134748121
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00001.safetensors",
+    "lm_head.scales": "model-00001.safetensors",
+    "lm_head.biases": "model-00001.safetensors",
+    "model.embed_tokens.weight": "model-00001.safetensors",
+    "model.embed_tokens.scales": "model-00001.safetensors",
+    "model.embed_tokens.biases": "model-00001.safetensors",
+    "model.norm.weight": "model-00001.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00002.safetensors",
+    "model.layers.0.mlp.down_proj.scales": "model-00002.safetensors",
+    "model.layers.0.mlp.down_proj.biases": "model-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.scales": "model-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.biases": "model-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00002.safetensors",
+    "model.layers.0.mlp.up_proj.scales": "model-00002.safetensors",
+    "model.layers.0.mlp.up_proj.biases": "model-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.indexer.k_norm.bias": "model-00002.safetensors",
+    "model.layers.0.self_attn.indexer.k_norm.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.indexer.weights_proj.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.indexer.wk.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.indexer.wq_b.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.kv_a_layernorm.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.kv_a_proj_with_mqa.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.kv_a_proj_with_mqa.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.kv_a_proj_with_mqa.biases": "model-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.biases": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_a_layernorm.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_a_proj.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_a_proj.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_a_proj.biases": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_b_proj.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_b_proj.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.q_b_proj.biases": "model-00002.safetensors",
+    "model.layers.0.self_attn.embed_q.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.embed_q.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.embed_q.biases": "model-00002.safetensors",
+    "model.layers.0.self_attn.unembed_out.weight": "model-00002.safetensors",
+    "model.layers.0.self_attn.unembed_out.scales": "model-00002.safetensors",
+    "model.layers.0.self_attn.unembed_out.biases": "model-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00003.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00003.safetensors",
+    "model.layers.1.mlp.down_proj.scales": "model-00003.safetensors",
+    "model.layers.1.mlp.down_proj.biases": "model-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.scales": "model-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.biases": "model-00003.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00003.safetensors",
+    "model.layers.1.mlp.up_proj.scales": "model-00003.safetensors",
+    "model.layers.1.mlp.up_proj.biases": "model-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.indexer.k_norm.bias": "model-00003.safetensors",
+    "model.layers.1.self_attn.indexer.k_norm.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.indexer.weights_proj.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.indexer.wk.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.indexer.wq_b.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.kv_a_layernorm.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.kv_a_proj_with_mqa.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.kv_a_proj_with_mqa.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.kv_a_proj_with_mqa.biases": "model-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.biases": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_a_layernorm.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_a_proj.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_a_proj.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_a_proj.biases": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_b_proj.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_b_proj.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.q_b_proj.biases": "model-00003.safetensors",
+    "model.layers.1.self_attn.embed_q.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.embed_q.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.embed_q.biases": "model-00003.safetensors",
+    "model.layers.1.self_attn.unembed_out.weight": "model-00003.safetensors",
+    "model.layers.1.self_attn.unembed_out.scales": "model-00003.safetensors",
+    "model.layers.1.self_attn.unembed_out.biases": "model-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00004.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00004.safetensors",
+    "model.layers.2.mlp.down_proj.scales": "model-00004.safetensors",
+    "model.layers.2.mlp.down_proj.biases": "model-00004.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00004.safetensors",
+    "model.layers.2.mlp.gate_proj.scales": "model-00004.safetensors",
+    "model.layers.2.mlp.gate_proj.biases": "model-00004.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00004.safetensors",
+    "model.layers.2.mlp.up_proj.scales": "model-00004.safetensors",
+    "model.layers.2.mlp.up_proj.biases": "model-00004.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.indexer.k_norm.bias": "model-00004.safetensors",
+    "model.layers.2.self_attn.indexer.k_norm.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.indexer.weights_proj.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.indexer.wk.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.indexer.wq_b.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.kv_a_layernorm.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.kv_a_proj_with_mqa.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.kv_a_proj_with_mqa.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.kv_a_proj_with_mqa.biases": "model-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.biases": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_a_layernorm.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_a_proj.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_a_proj.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_a_proj.biases": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_b_proj.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_b_proj.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.q_b_proj.biases": "model-00004.safetensors",
+    "model.layers.2.self_attn.embed_q.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.embed_q.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.embed_q.biases": "model-00004.safetensors",
+    "model.layers.2.self_attn.unembed_out.weight": "model-00004.safetensors",
+    "model.layers.2.self_attn.unembed_out.scales": "model-00004.safetensors",
+    "model.layers.2.self_attn.unembed_out.biases": "model-00004.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.gate.e_score_correction_bias": "model-00005.safetensors",
+    "model.layers.3.mlp.gate.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.down_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.down_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.down_proj.biases": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.gate_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.gate_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.gate_proj.biases": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.up_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.up_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.shared_experts.up_proj.biases": "model-00005.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.kv_a_layernorm.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.kv_a_proj_with_mqa.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.kv_a_proj_with_mqa.scales": "model-00005.safetensors",
+    "model.layers.3.self_attn.kv_a_proj_with_mqa.biases": "model-00005.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.o_proj.scales": "model-00005.safetensors",
+    "model.layers.3.self_attn.o_proj.biases": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_a_layernorm.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_a_proj.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_a_proj.scales": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_a_proj.biases": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_b_proj.weight": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_b_proj.scales": "model-00005.safetensors",
+    "model.layers.3.self_attn.q_b_proj.biases": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.gate_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.gate_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.gate_proj.biases": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.down_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.down_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.down_proj.biases": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.up_proj.weight": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.up_proj.scales": "model-00005.safetensors",
+    "model.layers.3.mlp.switch_mlp.up_proj.biases": "model-00005.safetensors",
+    "model.layers.3.self_attn.embed_q.weight": "model-00006.safetensors",
+    "model.layers.3.self_attn.embed_q.scales": "model-00006.safetensors",
+    "model.layers.3.self_attn.embed_q.biases": "model-00006.safetensors",
+    "model.layers.3.self_attn.unembed_out.weight": "model-00006.safetensors",
+    "model.layers.3.self_attn.unembed_out.scales": "model-00006.safetensors",
+    "model.layers.3.self_attn.unembed_out.biases": "model-00006.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.gate.e_score_correction_bias": "model-00007.safetensors",
+    "model.layers.4.mlp.gate.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.down_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.down_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.down_proj.biases": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.gate_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.gate_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.gate_proj.biases": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.up_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.up_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.shared_experts.up_proj.biases": "model-00007.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.kv_a_layernorm.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.kv_a_proj_with_mqa.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.kv_a_proj_with_mqa.scales": "model-00007.safetensors",
+    "model.layers.4.self_attn.kv_a_proj_with_mqa.biases": "model-00007.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.o_proj.scales": "model-00007.safetensors",
+    "model.layers.4.self_attn.o_proj.biases": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_a_layernorm.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_a_proj.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_a_proj.scales": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_a_proj.biases": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_b_proj.weight": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_b_proj.scales": "model-00007.safetensors",
+    "model.layers.4.self_attn.q_b_proj.biases": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.gate_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.gate_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.gate_proj.biases": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.down_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.down_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.down_proj.biases": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.up_proj.weight": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.up_proj.scales": "model-00007.safetensors",
+    "model.layers.4.mlp.switch_mlp.up_proj.biases": "model-00007.safetensors",
+    "model.layers.4.self_attn.embed_q.weight": "model-00008.safetensors",
+    "model.layers.4.self_attn.embed_q.scales": "model-00008.safetensors",
+    "model.layers.4.self_attn.embed_q.biases": "model-00008.safetensors",
+    "model.layers.4.self_attn.unembed_out.weight": "model-00008.safetensors",
+    "model.layers.4.self_attn.unembed_out.scales": "model-00008.safetensors",
+    "model.layers.4.self_attn.unembed_out.biases": "model-00008.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.gate.e_score_correction_bias": "model-00009.safetensors",
+    "model.layers.5.mlp.gate.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.down_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.down_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.down_proj.biases": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.gate_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.gate_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.gate_proj.biases": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.up_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.up_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.shared_experts.up_proj.biases": "model-00009.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.kv_a_layernorm.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.kv_a_proj_with_mqa.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.kv_a_proj_with_mqa.scales": "model-00009.safetensors",
+    "model.layers.5.self_attn.kv_a_proj_with_mqa.biases": "model-00009.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.o_proj.scales": "model-00009.safetensors",
+    "model.layers.5.self_attn.o_proj.biases": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_a_layernorm.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_a_proj.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_a_proj.scales": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_a_proj.biases": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_b_proj.weight": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_b_proj.scales": "model-00009.safetensors",
+    "model.layers.5.self_attn.q_b_proj.biases": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.gate_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.gate_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.gate_proj.biases": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.down_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.down_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.down_proj.biases": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.up_proj.weight": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.up_proj.scales": "model-00009.safetensors",
+    "model.layers.5.mlp.switch_mlp.up_proj.biases": "model-00009.safetensors",
+    "model.layers.5.self_attn.embed_q.weight": "model-00010.safetensors",
+    "model.layers.5.self_attn.embed_q.scales": "model-00010.safetensors",
+    "model.layers.5.self_attn.embed_q.biases": "model-00010.safetensors",
+    "model.layers.5.self_attn.unembed_out.weight": "model-00010.safetensors",
+    "model.layers.5.self_attn.unembed_out.scales": "model-00010.safetensors",
+    "model.layers.5.self_attn.unembed_out.biases": "model-00010.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.gate.e_score_correction_bias": "model-00011.safetensors",
+    "model.layers.6.mlp.gate.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.down_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.down_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.down_proj.biases": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.gate_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.gate_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.gate_proj.biases": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.up_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.up_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.shared_experts.up_proj.biases": "model-00011.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.indexer.k_norm.bias": "model-00011.safetensors",
+    "model.layers.6.self_attn.indexer.k_norm.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.indexer.weights_proj.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.indexer.wk.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.indexer.wq_b.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.kv_a_layernorm.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.kv_a_proj_with_mqa.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.kv_a_proj_with_mqa.scales": "model-00011.safetensors",
+    "model.layers.6.self_attn.kv_a_proj_with_mqa.biases": "model-00011.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.o_proj.scales": "model-00011.safetensors",
+    "model.layers.6.self_attn.o_proj.biases": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_a_layernorm.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_a_proj.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_a_proj.scales": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_a_proj.biases": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_b_proj.weight": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_b_proj.scales": "model-00011.safetensors",
+    "model.layers.6.self_attn.q_b_proj.biases": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.gate_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.gate_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.gate_proj.biases": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.down_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.down_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.down_proj.biases": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.up_proj.weight": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.up_proj.scales": "model-00011.safetensors",
+    "model.layers.6.mlp.switch_mlp.up_proj.biases": "model-00011.safetensors",
+    "model.layers.6.self_attn.embed_q.weight": "model-00012.safetensors",
+    "model.layers.6.self_attn.embed_q.scales": "model-00012.safetensors",
+    "model.layers.6.self_attn.embed_q.biases": "model-00012.safetensors",
+    "model.layers.6.self_attn.unembed_out.weight": "model-00012.safetensors",
+    "model.layers.6.self_attn.unembed_out.scales": "model-00012.safetensors",
+    "model.layers.6.self_attn.unembed_out.biases": "model-00012.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.gate.e_score_correction_bias": "model-00013.safetensors",
+    "model.layers.7.mlp.gate.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.down_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.down_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.down_proj.biases": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.gate_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.gate_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.gate_proj.biases": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.up_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.up_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.shared_experts.up_proj.biases": "model-00013.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.kv_a_layernorm.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.kv_a_proj_with_mqa.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.kv_a_proj_with_mqa.scales": "model-00013.safetensors",
+    "model.layers.7.self_attn.kv_a_proj_with_mqa.biases": "model-00013.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.o_proj.scales": "model-00013.safetensors",
+    "model.layers.7.self_attn.o_proj.biases": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_a_layernorm.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_a_proj.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_a_proj.scales": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_a_proj.biases": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_b_proj.weight": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_b_proj.scales": "model-00013.safetensors",
+    "model.layers.7.self_attn.q_b_proj.biases": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.gate_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.gate_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.gate_proj.biases": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.down_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.down_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.down_proj.biases": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.up_proj.weight": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.up_proj.scales": "model-00013.safetensors",
+    "model.layers.7.mlp.switch_mlp.up_proj.biases": "model-00013.safetensors",
+    "model.layers.7.self_attn.embed_q.weight": "model-00014.safetensors",
+    "model.layers.7.self_attn.embed_q.scales": "model-00014.safetensors",
+    "model.layers.7.self_attn.embed_q.biases": "model-00014.safetensors",
+    "model.layers.7.self_attn.unembed_out.weight": "model-00014.safetensors",
+    "model.layers.7.self_attn.unembed_out.scales": "model-00014.safetensors",
+    "model.layers.7.self_attn.unembed_out.biases": "model-00014.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.gate.e_score_correction_bias": "model-00015.safetensors",
+    "model.layers.8.mlp.gate.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.down_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.down_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.down_proj.biases": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.gate_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.gate_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.gate_proj.biases": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.up_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.up_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.shared_experts.up_proj.biases": "model-00015.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.kv_a_layernorm.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.kv_a_proj_with_mqa.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.kv_a_proj_with_mqa.scales": "model-00015.safetensors",
+    "model.layers.8.self_attn.kv_a_proj_with_mqa.biases": "model-00015.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.o_proj.scales": "model-00015.safetensors",
+    "model.layers.8.self_attn.o_proj.biases": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_a_layernorm.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_a_proj.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_a_proj.scales": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_a_proj.biases": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_b_proj.weight": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_b_proj.scales": "model-00015.safetensors",
+    "model.layers.8.self_attn.q_b_proj.biases": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.gate_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.gate_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.gate_proj.biases": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.down_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.down_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.down_proj.biases": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.up_proj.weight": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.up_proj.scales": "model-00015.safetensors",
+    "model.layers.8.mlp.switch_mlp.up_proj.biases": "model-00015.safetensors",
+    "model.layers.8.self_attn.embed_q.weight": "model-00016.safetensors",
+    "model.layers.8.self_attn.embed_q.scales": "model-00016.safetensors",
+    "model.layers.8.self_attn.embed_q.biases": "model-00016.safetensors",
+    "model.layers.8.self_attn.unembed_out.weight": "model-00016.safetensors",
+    "model.layers.8.self_attn.unembed_out.scales": "model-00016.safetensors",
+    "model.layers.8.self_attn.unembed_out.biases": "model-00016.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.gate.e_score_correction_bias": "model-00017.safetensors",
+    "model.layers.9.mlp.gate.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.down_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.down_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.down_proj.biases": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.gate_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.gate_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.gate_proj.biases": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.up_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.up_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.shared_experts.up_proj.biases": "model-00017.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.kv_a_layernorm.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.kv_a_proj_with_mqa.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.kv_a_proj_with_mqa.scales": "model-00017.safetensors",
+    "model.layers.9.self_attn.kv_a_proj_with_mqa.biases": "model-00017.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.o_proj.scales": "model-00017.safetensors",
+    "model.layers.9.self_attn.o_proj.biases": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_a_layernorm.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_a_proj.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_a_proj.scales": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_a_proj.biases": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_b_proj.weight": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_b_proj.scales": "model-00017.safetensors",
+    "model.layers.9.self_attn.q_b_proj.biases": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.gate_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.gate_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.gate_proj.biases": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.down_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.down_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.down_proj.biases": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.up_proj.weight": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.up_proj.scales": "model-00017.safetensors",
+    "model.layers.9.mlp.switch_mlp.up_proj.biases": "model-00017.safetensors",
+    "model.layers.9.self_attn.embed_q.weight": "model-00018.safetensors",
+    "model.layers.9.self_attn.embed_q.scales": "model-00018.safetensors",
+    "model.layers.9.self_attn.embed_q.biases": "model-00018.safetensors",
+    "model.layers.9.self_attn.unembed_out.weight": "model-00018.safetensors",
+    "model.layers.9.self_attn.unembed_out.scales": "model-00018.safetensors",
+    "model.layers.9.self_attn.unembed_out.biases": "model-00018.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.gate.e_score_correction_bias": "model-00019.safetensors",
+    "model.layers.10.mlp.gate.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.down_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.down_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.down_proj.biases": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.gate_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.gate_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.gate_proj.biases": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.up_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.up_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.shared_experts.up_proj.biases": "model-00019.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.indexer.k_norm.bias": "model-00019.safetensors",
+    "model.layers.10.self_attn.indexer.k_norm.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.indexer.weights_proj.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.indexer.wk.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.indexer.wq_b.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.kv_a_layernorm.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.kv_a_proj_with_mqa.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.kv_a_proj_with_mqa.scales": "model-00019.safetensors",
+    "model.layers.10.self_attn.kv_a_proj_with_mqa.biases": "model-00019.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.o_proj.scales": "model-00019.safetensors",
+    "model.layers.10.self_attn.o_proj.biases": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_a_layernorm.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_a_proj.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_a_proj.scales": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_a_proj.biases": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_b_proj.weight": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_b_proj.scales": "model-00019.safetensors",
+    "model.layers.10.self_attn.q_b_proj.biases": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.gate_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.gate_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.gate_proj.biases": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.down_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.down_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.down_proj.biases": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.up_proj.weight": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.up_proj.scales": "model-00019.safetensors",
+    "model.layers.10.mlp.switch_mlp.up_proj.biases": "model-00019.safetensors",
+    "model.layers.10.self_attn.embed_q.weight": "model-00020.safetensors",
+    "model.layers.10.self_attn.embed_q.scales": "model-00020.safetensors",
+    "model.layers.10.self_attn.embed_q.biases": "model-00020.safetensors",
+    "model.layers.10.self_attn.unembed_out.weight": "model-00020.safetensors",
+    "model.layers.10.self_attn.unembed_out.scales": "model-00020.safetensors",
+    "model.layers.10.self_attn.unembed_out.biases": "model-00020.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.gate.e_score_correction_bias": "model-00021.safetensors",
+    "model.layers.11.mlp.gate.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.down_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.down_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.down_proj.biases": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.gate_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.gate_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.gate_proj.biases": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.up_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.up_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.shared_experts.up_proj.biases": "model-00021.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.kv_a_layernorm.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.kv_a_proj_with_mqa.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.kv_a_proj_with_mqa.scales": "model-00021.safetensors",
+    "model.layers.11.self_attn.kv_a_proj_with_mqa.biases": "model-00021.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.o_proj.scales": "model-00021.safetensors",
+    "model.layers.11.self_attn.o_proj.biases": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_a_layernorm.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_a_proj.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_a_proj.scales": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_a_proj.biases": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_b_proj.weight": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_b_proj.scales": "model-00021.safetensors",
+    "model.layers.11.self_attn.q_b_proj.biases": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.gate_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.gate_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.gate_proj.biases": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.down_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.down_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.down_proj.biases": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.up_proj.weight": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.up_proj.scales": "model-00021.safetensors",
+    "model.layers.11.mlp.switch_mlp.up_proj.biases": "model-00021.safetensors",
+    "model.layers.11.self_attn.embed_q.weight": "model-00022.safetensors",
+    "model.layers.11.self_attn.embed_q.scales": "model-00022.safetensors",
+    "model.layers.11.self_attn.embed_q.biases": "model-00022.safetensors",
+    "model.layers.11.self_attn.unembed_out.weight": "model-00022.safetensors",
+    "model.layers.11.self_attn.unembed_out.scales": "model-00022.safetensors",
+    "model.layers.11.self_attn.unembed_out.biases": "model-00022.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.gate.e_score_correction_bias": "model-00023.safetensors",
+    "model.layers.12.mlp.gate.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.down_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.down_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.down_proj.biases": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.gate_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.gate_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.gate_proj.biases": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.up_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.up_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.shared_experts.up_proj.biases": "model-00023.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.kv_a_layernorm.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.kv_a_proj_with_mqa.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.kv_a_proj_with_mqa.scales": "model-00023.safetensors",
+    "model.layers.12.self_attn.kv_a_proj_with_mqa.biases": "model-00023.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.o_proj.scales": "model-00023.safetensors",
+    "model.layers.12.self_attn.o_proj.biases": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_a_layernorm.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_a_proj.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_a_proj.scales": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_a_proj.biases": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_b_proj.weight": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_b_proj.scales": "model-00023.safetensors",
+    "model.layers.12.self_attn.q_b_proj.biases": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.gate_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.gate_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.gate_proj.biases": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.down_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.down_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.down_proj.biases": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.up_proj.weight": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.up_proj.scales": "model-00023.safetensors",
+    "model.layers.12.mlp.switch_mlp.up_proj.biases": "model-00023.safetensors",
+    "model.layers.12.self_attn.embed_q.weight": "model-00024.safetensors",
+    "model.layers.12.self_attn.embed_q.scales": "model-00024.safetensors",
+    "model.layers.12.self_attn.embed_q.biases": "model-00024.safetensors",
+    "model.layers.12.self_attn.unembed_out.weight": "model-00024.safetensors",
+    "model.layers.12.self_attn.unembed_out.scales": "model-00024.safetensors",
+    "model.layers.12.self_attn.unembed_out.biases": "model-00024.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.gate.e_score_correction_bias": "model-00025.safetensors",
+    "model.layers.13.mlp.gate.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.down_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.down_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.down_proj.biases": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.gate_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.gate_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.gate_proj.biases": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.up_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.up_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.shared_experts.up_proj.biases": "model-00025.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.kv_a_layernorm.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.kv_a_proj_with_mqa.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.kv_a_proj_with_mqa.scales": "model-00025.safetensors",
+    "model.layers.13.self_attn.kv_a_proj_with_mqa.biases": "model-00025.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.o_proj.scales": "model-00025.safetensors",
+    "model.layers.13.self_attn.o_proj.biases": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_a_layernorm.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_a_proj.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_a_proj.scales": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_a_proj.biases": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_b_proj.weight": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_b_proj.scales": "model-00025.safetensors",
+    "model.layers.13.self_attn.q_b_proj.biases": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.gate_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.gate_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.gate_proj.biases": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.down_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.down_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.down_proj.biases": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.up_proj.weight": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.up_proj.scales": "model-00025.safetensors",
+    "model.layers.13.mlp.switch_mlp.up_proj.biases": "model-00025.safetensors",
+    "model.layers.13.self_attn.embed_q.weight": "model-00026.safetensors",
+    "model.layers.13.self_attn.embed_q.scales": "model-00026.safetensors",
+    "model.layers.13.self_attn.embed_q.biases": "model-00026.safetensors",
+    "model.layers.13.self_attn.unembed_out.weight": "model-00026.safetensors",
+    "model.layers.13.self_attn.unembed_out.scales": "model-00026.safetensors",
+    "model.layers.13.self_attn.unembed_out.biases": "model-00026.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.gate.e_score_correction_bias": "model-00027.safetensors",
+    "model.layers.14.mlp.gate.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.down_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.down_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.down_proj.biases": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.gate_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.gate_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.gate_proj.biases": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.up_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.up_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.shared_experts.up_proj.biases": "model-00027.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.indexer.k_norm.bias": "model-00027.safetensors",
+    "model.layers.14.self_attn.indexer.k_norm.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.indexer.weights_proj.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.indexer.wk.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.indexer.wq_b.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.kv_a_layernorm.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.kv_a_proj_with_mqa.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.kv_a_proj_with_mqa.scales": "model-00027.safetensors",
+    "model.layers.14.self_attn.kv_a_proj_with_mqa.biases": "model-00027.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.o_proj.scales": "model-00027.safetensors",
+    "model.layers.14.self_attn.o_proj.biases": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_a_layernorm.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_a_proj.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_a_proj.scales": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_a_proj.biases": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_b_proj.weight": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_b_proj.scales": "model-00027.safetensors",
+    "model.layers.14.self_attn.q_b_proj.biases": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.gate_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.gate_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.gate_proj.biases": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.down_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.down_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.down_proj.biases": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.up_proj.weight": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.up_proj.scales": "model-00027.safetensors",
+    "model.layers.14.mlp.switch_mlp.up_proj.biases": "model-00027.safetensors",
+    "model.layers.14.self_attn.embed_q.weight": "model-00028.safetensors",
+    "model.layers.14.self_attn.embed_q.scales": "model-00028.safetensors",
+    "model.layers.14.self_attn.embed_q.biases": "model-00028.safetensors",
+    "model.layers.14.self_attn.unembed_out.weight": "model-00028.safetensors",
+    "model.layers.14.self_attn.unembed_out.scales": "model-00028.safetensors",
+    "model.layers.14.self_attn.unembed_out.biases": "model-00028.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.gate.e_score_correction_bias": "model-00029.safetensors",
+    "model.layers.15.mlp.gate.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.down_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.down_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.down_proj.biases": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.gate_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.gate_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.gate_proj.biases": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.up_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.up_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.shared_experts.up_proj.biases": "model-00029.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.kv_a_layernorm.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.kv_a_proj_with_mqa.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.kv_a_proj_with_mqa.scales": "model-00029.safetensors",
+    "model.layers.15.self_attn.kv_a_proj_with_mqa.biases": "model-00029.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.o_proj.scales": "model-00029.safetensors",
+    "model.layers.15.self_attn.o_proj.biases": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_a_layernorm.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_a_proj.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_a_proj.scales": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_a_proj.biases": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_b_proj.weight": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_b_proj.scales": "model-00029.safetensors",
+    "model.layers.15.self_attn.q_b_proj.biases": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.gate_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.gate_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.gate_proj.biases": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.down_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.down_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.down_proj.biases": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.up_proj.weight": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.up_proj.scales": "model-00029.safetensors",
+    "model.layers.15.mlp.switch_mlp.up_proj.biases": "model-00029.safetensors",
+    "model.layers.15.self_attn.embed_q.weight": "model-00030.safetensors",
+    "model.layers.15.self_attn.embed_q.scales": "model-00030.safetensors",
+    "model.layers.15.self_attn.embed_q.biases": "model-00030.safetensors",
+    "model.layers.15.self_attn.unembed_out.weight": "model-00030.safetensors",
+    "model.layers.15.self_attn.unembed_out.scales": "model-00030.safetensors",
+    "model.layers.15.self_attn.unembed_out.biases": "model-00030.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.gate.e_score_correction_bias": "model-00031.safetensors",
+    "model.layers.16.mlp.gate.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.down_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.down_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.down_proj.biases": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.gate_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.gate_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.gate_proj.biases": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.up_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.up_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.shared_experts.up_proj.biases": "model-00031.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.kv_a_layernorm.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.kv_a_proj_with_mqa.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.kv_a_proj_with_mqa.scales": "model-00031.safetensors",
+    "model.layers.16.self_attn.kv_a_proj_with_mqa.biases": "model-00031.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.o_proj.scales": "model-00031.safetensors",
+    "model.layers.16.self_attn.o_proj.biases": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_a_layernorm.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_a_proj.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_a_proj.scales": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_a_proj.biases": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_b_proj.weight": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_b_proj.scales": "model-00031.safetensors",
+    "model.layers.16.self_attn.q_b_proj.biases": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.gate_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.gate_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.gate_proj.biases": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.down_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.down_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.down_proj.biases": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.up_proj.weight": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.up_proj.scales": "model-00031.safetensors",
+    "model.layers.16.mlp.switch_mlp.up_proj.biases": "model-00031.safetensors",
+    "model.layers.16.self_attn.embed_q.weight": "model-00032.safetensors",
+    "model.layers.16.self_attn.embed_q.scales": "model-00032.safetensors",
+    "model.layers.16.self_attn.embed_q.biases": "model-00032.safetensors",
+    "model.layers.16.self_attn.unembed_out.weight": "model-00032.safetensors",
+    "model.layers.16.self_attn.unembed_out.scales": "model-00032.safetensors",
+    "model.layers.16.self_attn.unembed_out.biases": "model-00032.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.gate.e_score_correction_bias": "model-00033.safetensors",
+    "model.layers.17.mlp.gate.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.down_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.down_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.down_proj.biases": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.gate_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.gate_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.gate_proj.biases": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.up_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.up_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.shared_experts.up_proj.biases": "model-00033.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.kv_a_layernorm.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.kv_a_proj_with_mqa.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.kv_a_proj_with_mqa.scales": "model-00033.safetensors",
+    "model.layers.17.self_attn.kv_a_proj_with_mqa.biases": "model-00033.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.o_proj.scales": "model-00033.safetensors",
+    "model.layers.17.self_attn.o_proj.biases": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_a_layernorm.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_a_proj.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_a_proj.scales": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_a_proj.biases": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_b_proj.weight": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_b_proj.scales": "model-00033.safetensors",
+    "model.layers.17.self_attn.q_b_proj.biases": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.gate_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.gate_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.gate_proj.biases": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.down_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.down_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.down_proj.biases": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.up_proj.weight": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.up_proj.scales": "model-00033.safetensors",
+    "model.layers.17.mlp.switch_mlp.up_proj.biases": "model-00033.safetensors",
+    "model.layers.17.self_attn.embed_q.weight": "model-00034.safetensors",
+    "model.layers.17.self_attn.embed_q.scales": "model-00034.safetensors",
+    "model.layers.17.self_attn.embed_q.biases": "model-00034.safetensors",
+    "model.layers.17.self_attn.unembed_out.weight": "model-00034.safetensors",
+    "model.layers.17.self_attn.unembed_out.scales": "model-00034.safetensors",
+    "model.layers.17.self_attn.unembed_out.biases": "model-00034.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.gate.e_score_correction_bias": "model-00035.safetensors",
+    "model.layers.18.mlp.gate.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.down_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.down_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.down_proj.biases": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.gate_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.gate_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.gate_proj.biases": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.up_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.up_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.shared_experts.up_proj.biases": "model-00035.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.indexer.k_norm.bias": "model-00035.safetensors",
+    "model.layers.18.self_attn.indexer.k_norm.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.indexer.weights_proj.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.indexer.wk.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.indexer.wq_b.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.kv_a_layernorm.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.kv_a_proj_with_mqa.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.kv_a_proj_with_mqa.scales": "model-00035.safetensors",
+    "model.layers.18.self_attn.kv_a_proj_with_mqa.biases": "model-00035.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.o_proj.scales": "model-00035.safetensors",
+    "model.layers.18.self_attn.o_proj.biases": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_a_layernorm.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_a_proj.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_a_proj.scales": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_a_proj.biases": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_b_proj.weight": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_b_proj.scales": "model-00035.safetensors",
+    "model.layers.18.self_attn.q_b_proj.biases": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.gate_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.gate_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.gate_proj.biases": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.down_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.down_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.down_proj.biases": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.up_proj.weight": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.up_proj.scales": "model-00035.safetensors",
+    "model.layers.18.mlp.switch_mlp.up_proj.biases": "model-00035.safetensors",
+    "model.layers.18.self_attn.embed_q.weight": "model-00036.safetensors",
+    "model.layers.18.self_attn.embed_q.scales": "model-00036.safetensors",
+    "model.layers.18.self_attn.embed_q.biases": "model-00036.safetensors",
+    "model.layers.18.self_attn.unembed_out.weight": "model-00036.safetensors",
+    "model.layers.18.self_attn.unembed_out.scales": "model-00036.safetensors",
+    "model.layers.18.self_attn.unembed_out.biases": "model-00036.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.gate.e_score_correction_bias": "model-00037.safetensors",
+    "model.layers.19.mlp.gate.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.down_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.down_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.down_proj.biases": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.gate_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.gate_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.gate_proj.biases": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.up_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.up_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.shared_experts.up_proj.biases": "model-00037.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.kv_a_layernorm.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.kv_a_proj_with_mqa.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.kv_a_proj_with_mqa.scales": "model-00037.safetensors",
+    "model.layers.19.self_attn.kv_a_proj_with_mqa.biases": "model-00037.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.o_proj.scales": "model-00037.safetensors",
+    "model.layers.19.self_attn.o_proj.biases": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_a_layernorm.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_a_proj.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_a_proj.scales": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_a_proj.biases": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_b_proj.weight": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_b_proj.scales": "model-00037.safetensors",
+    "model.layers.19.self_attn.q_b_proj.biases": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.gate_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.gate_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.gate_proj.biases": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.down_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.down_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.down_proj.biases": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.up_proj.weight": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.up_proj.scales": "model-00037.safetensors",
+    "model.layers.19.mlp.switch_mlp.up_proj.biases": "model-00037.safetensors",
+    "model.layers.19.self_attn.embed_q.weight": "model-00038.safetensors",
+    "model.layers.19.self_attn.embed_q.scales": "model-00038.safetensors",
+    "model.layers.19.self_attn.embed_q.biases": "model-00038.safetensors",
+    "model.layers.19.self_attn.unembed_out.weight": "model-00038.safetensors",
+    "model.layers.19.self_attn.unembed_out.scales": "model-00038.safetensors",
+    "model.layers.19.self_attn.unembed_out.biases": "model-00038.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.gate.e_score_correction_bias": "model-00039.safetensors",
+    "model.layers.20.mlp.gate.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.down_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.down_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.down_proj.biases": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.gate_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.gate_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.gate_proj.biases": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.up_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.up_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.shared_experts.up_proj.biases": "model-00039.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.kv_a_layernorm.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.kv_a_proj_with_mqa.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.kv_a_proj_with_mqa.scales": "model-00039.safetensors",
+    "model.layers.20.self_attn.kv_a_proj_with_mqa.biases": "model-00039.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.o_proj.scales": "model-00039.safetensors",
+    "model.layers.20.self_attn.o_proj.biases": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_a_layernorm.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_a_proj.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_a_proj.scales": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_a_proj.biases": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_b_proj.weight": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_b_proj.scales": "model-00039.safetensors",
+    "model.layers.20.self_attn.q_b_proj.biases": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.gate_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.gate_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.gate_proj.biases": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.down_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.down_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.down_proj.biases": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.up_proj.weight": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.up_proj.scales": "model-00039.safetensors",
+    "model.layers.20.mlp.switch_mlp.up_proj.biases": "model-00039.safetensors",
+    "model.layers.20.self_attn.embed_q.weight": "model-00040.safetensors",
+    "model.layers.20.self_attn.embed_q.scales": "model-00040.safetensors",
+    "model.layers.20.self_attn.embed_q.biases": "model-00040.safetensors",
+    "model.layers.20.self_attn.unembed_out.weight": "model-00040.safetensors",
+    "model.layers.20.self_attn.unembed_out.scales": "model-00040.safetensors",
+    "model.layers.20.self_attn.unembed_out.biases": "model-00040.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.gate.e_score_correction_bias": "model-00041.safetensors",
+    "model.layers.21.mlp.gate.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.down_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.down_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.down_proj.biases": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.gate_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.gate_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.gate_proj.biases": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.up_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.up_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.shared_experts.up_proj.biases": "model-00041.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.kv_a_layernorm.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.kv_a_proj_with_mqa.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.kv_a_proj_with_mqa.scales": "model-00041.safetensors",
+    "model.layers.21.self_attn.kv_a_proj_with_mqa.biases": "model-00041.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.o_proj.scales": "model-00041.safetensors",
+    "model.layers.21.self_attn.o_proj.biases": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_a_layernorm.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_a_proj.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_a_proj.scales": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_a_proj.biases": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_b_proj.weight": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_b_proj.scales": "model-00041.safetensors",
+    "model.layers.21.self_attn.q_b_proj.biases": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.gate_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.gate_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.gate_proj.biases": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.down_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.down_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.down_proj.biases": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.up_proj.weight": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.up_proj.scales": "model-00041.safetensors",
+    "model.layers.21.mlp.switch_mlp.up_proj.biases": "model-00041.safetensors",
+    "model.layers.21.self_attn.embed_q.weight": "model-00042.safetensors",
+    "model.layers.21.self_attn.embed_q.scales": "model-00042.safetensors",
+    "model.layers.21.self_attn.embed_q.biases": "model-00042.safetensors",
+    "model.layers.21.self_attn.unembed_out.weight": "model-00042.safetensors",
+    "model.layers.21.self_attn.unembed_out.scales": "model-00042.safetensors",
+    "model.layers.21.self_attn.unembed_out.biases": "model-00042.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.gate.e_score_correction_bias": "model-00043.safetensors",
+    "model.layers.22.mlp.gate.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.down_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.down_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.down_proj.biases": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.gate_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.gate_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.gate_proj.biases": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.up_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.up_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.shared_experts.up_proj.biases": "model-00043.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.indexer.k_norm.bias": "model-00043.safetensors",
+    "model.layers.22.self_attn.indexer.k_norm.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.indexer.weights_proj.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.indexer.wk.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.indexer.wq_b.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.kv_a_layernorm.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.kv_a_proj_with_mqa.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.kv_a_proj_with_mqa.scales": "model-00043.safetensors",
+    "model.layers.22.self_attn.kv_a_proj_with_mqa.biases": "model-00043.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.o_proj.scales": "model-00043.safetensors",
+    "model.layers.22.self_attn.o_proj.biases": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_a_layernorm.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_a_proj.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_a_proj.scales": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_a_proj.biases": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_b_proj.weight": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_b_proj.scales": "model-00043.safetensors",
+    "model.layers.22.self_attn.q_b_proj.biases": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.gate_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.gate_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.gate_proj.biases": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.down_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.down_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.down_proj.biases": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.up_proj.weight": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.up_proj.scales": "model-00043.safetensors",
+    "model.layers.22.mlp.switch_mlp.up_proj.biases": "model-00043.safetensors",
+    "model.layers.22.self_attn.embed_q.weight": "model-00044.safetensors",
+    "model.layers.22.self_attn.embed_q.scales": "model-00044.safetensors",
+    "model.layers.22.self_attn.embed_q.biases": "model-00044.safetensors",
+    "model.layers.22.self_attn.unembed_out.weight": "model-00044.safetensors",
+    "model.layers.22.self_attn.unembed_out.scales": "model-00044.safetensors",
+    "model.layers.22.self_attn.unembed_out.biases": "model-00044.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.gate.e_score_correction_bias": "model-00045.safetensors",
+    "model.layers.23.mlp.gate.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.down_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.down_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.down_proj.biases": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.gate_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.gate_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.gate_proj.biases": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.up_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.up_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.shared_experts.up_proj.biases": "model-00045.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.kv_a_layernorm.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.kv_a_proj_with_mqa.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.kv_a_proj_with_mqa.scales": "model-00045.safetensors",
+    "model.layers.23.self_attn.kv_a_proj_with_mqa.biases": "model-00045.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.o_proj.scales": "model-00045.safetensors",
+    "model.layers.23.self_attn.o_proj.biases": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_a_layernorm.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_a_proj.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_a_proj.scales": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_a_proj.biases": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_b_proj.weight": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_b_proj.scales": "model-00045.safetensors",
+    "model.layers.23.self_attn.q_b_proj.biases": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.gate_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.gate_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.gate_proj.biases": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.down_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.down_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.down_proj.biases": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.up_proj.weight": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.up_proj.scales": "model-00045.safetensors",
+    "model.layers.23.mlp.switch_mlp.up_proj.biases": "model-00045.safetensors",
+    "model.layers.23.self_attn.embed_q.weight": "model-00046.safetensors",
+    "model.layers.23.self_attn.embed_q.scales": "model-00046.safetensors",
+    "model.layers.23.self_attn.embed_q.biases": "model-00046.safetensors",
+    "model.layers.23.self_attn.unembed_out.weight": "model-00046.safetensors",
+    "model.layers.23.self_attn.unembed_out.scales": "model-00046.safetensors",
+    "model.layers.23.self_attn.unembed_out.biases": "model-00046.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.gate.e_score_correction_bias": "model-00047.safetensors",
+    "model.layers.24.mlp.gate.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.down_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.down_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.down_proj.biases": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.gate_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.gate_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.gate_proj.biases": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.up_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.up_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.shared_experts.up_proj.biases": "model-00047.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.kv_a_layernorm.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.kv_a_proj_with_mqa.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.kv_a_proj_with_mqa.scales": "model-00047.safetensors",
+    "model.layers.24.self_attn.kv_a_proj_with_mqa.biases": "model-00047.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.o_proj.scales": "model-00047.safetensors",
+    "model.layers.24.self_attn.o_proj.biases": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_a_layernorm.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_a_proj.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_a_proj.scales": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_a_proj.biases": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_b_proj.weight": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_b_proj.scales": "model-00047.safetensors",
+    "model.layers.24.self_attn.q_b_proj.biases": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.gate_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.gate_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.gate_proj.biases": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.down_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.down_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.down_proj.biases": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.up_proj.weight": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.up_proj.scales": "model-00047.safetensors",
+    "model.layers.24.mlp.switch_mlp.up_proj.biases": "model-00047.safetensors",
+    "model.layers.24.self_attn.embed_q.weight": "model-00048.safetensors",
+    "model.layers.24.self_attn.embed_q.scales": "model-00048.safetensors",
+    "model.layers.24.self_attn.embed_q.biases": "model-00048.safetensors",
+    "model.layers.24.self_attn.unembed_out.weight": "model-00048.safetensors",
+    "model.layers.24.self_attn.unembed_out.scales": "model-00048.safetensors",
+    "model.layers.24.self_attn.unembed_out.biases": "model-00048.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.gate.e_score_correction_bias": "model-00049.safetensors",
+    "model.layers.25.mlp.gate.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.down_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.down_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.down_proj.biases": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.gate_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.gate_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.gate_proj.biases": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.up_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.up_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.shared_experts.up_proj.biases": "model-00049.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.kv_a_layernorm.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.kv_a_proj_with_mqa.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.kv_a_proj_with_mqa.scales": "model-00049.safetensors",
+    "model.layers.25.self_attn.kv_a_proj_with_mqa.biases": "model-00049.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.o_proj.scales": "model-00049.safetensors",
+    "model.layers.25.self_attn.o_proj.biases": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_a_layernorm.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_a_proj.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_a_proj.scales": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_a_proj.biases": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_b_proj.weight": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_b_proj.scales": "model-00049.safetensors",
+    "model.layers.25.self_attn.q_b_proj.biases": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.gate_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.gate_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.gate_proj.biases": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.down_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.down_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.down_proj.biases": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.up_proj.weight": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.up_proj.scales": "model-00049.safetensors",
+    "model.layers.25.mlp.switch_mlp.up_proj.biases": "model-00049.safetensors",
+    "model.layers.25.self_attn.embed_q.weight": "model-00050.safetensors",
+    "model.layers.25.self_attn.embed_q.scales": "model-00050.safetensors",
+    "model.layers.25.self_attn.embed_q.biases": "model-00050.safetensors",
+    "model.layers.25.self_attn.unembed_out.weight": "model-00050.safetensors",
+    "model.layers.25.self_attn.unembed_out.scales": "model-00050.safetensors",
+    "model.layers.25.self_attn.unembed_out.biases": "model-00050.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.gate.e_score_correction_bias": "model-00051.safetensors",
+    "model.layers.26.mlp.gate.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.down_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.down_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.down_proj.biases": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.gate_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.gate_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.gate_proj.biases": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.up_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.up_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.shared_experts.up_proj.biases": "model-00051.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.indexer.k_norm.bias": "model-00051.safetensors",
+    "model.layers.26.self_attn.indexer.k_norm.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.indexer.weights_proj.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.indexer.wk.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.indexer.wq_b.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.kv_a_layernorm.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.kv_a_proj_with_mqa.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.kv_a_proj_with_mqa.scales": "model-00051.safetensors",
+    "model.layers.26.self_attn.kv_a_proj_with_mqa.biases": "model-00051.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.o_proj.scales": "model-00051.safetensors",
+    "model.layers.26.self_attn.o_proj.biases": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_a_layernorm.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_a_proj.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_a_proj.scales": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_a_proj.biases": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_b_proj.weight": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_b_proj.scales": "model-00051.safetensors",
+    "model.layers.26.self_attn.q_b_proj.biases": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.gate_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.gate_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.gate_proj.biases": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.down_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.down_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.down_proj.biases": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.up_proj.weight": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.up_proj.scales": "model-00051.safetensors",
+    "model.layers.26.mlp.switch_mlp.up_proj.biases": "model-00051.safetensors",
+    "model.layers.26.self_attn.embed_q.weight": "model-00052.safetensors",
+    "model.layers.26.self_attn.embed_q.scales": "model-00052.safetensors",
+    "model.layers.26.self_attn.embed_q.biases": "model-00052.safetensors",
+    "model.layers.26.self_attn.unembed_out.weight": "model-00052.safetensors",
+    "model.layers.26.self_attn.unembed_out.scales": "model-00052.safetensors",
+    "model.layers.26.self_attn.unembed_out.biases": "model-00052.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.gate.e_score_correction_bias": "model-00053.safetensors",
+    "model.layers.27.mlp.gate.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.down_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.down_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.down_proj.biases": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.gate_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.gate_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.gate_proj.biases": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.up_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.up_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.shared_experts.up_proj.biases": "model-00053.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.kv_a_layernorm.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.kv_a_proj_with_mqa.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.kv_a_proj_with_mqa.scales": "model-00053.safetensors",
+    "model.layers.27.self_attn.kv_a_proj_with_mqa.biases": "model-00053.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.o_proj.scales": "model-00053.safetensors",
+    "model.layers.27.self_attn.o_proj.biases": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_a_layernorm.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_a_proj.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_a_proj.scales": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_a_proj.biases": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_b_proj.weight": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_b_proj.scales": "model-00053.safetensors",
+    "model.layers.27.self_attn.q_b_proj.biases": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.gate_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.gate_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.gate_proj.biases": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.down_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.down_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.down_proj.biases": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.up_proj.weight": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.up_proj.scales": "model-00053.safetensors",
+    "model.layers.27.mlp.switch_mlp.up_proj.biases": "model-00053.safetensors",
+    "model.layers.27.self_attn.embed_q.weight": "model-00054.safetensors",
+    "model.layers.27.self_attn.embed_q.scales": "model-00054.safetensors",
+    "model.layers.27.self_attn.embed_q.biases": "model-00054.safetensors",
+    "model.layers.27.self_attn.unembed_out.weight": "model-00054.safetensors",
+    "model.layers.27.self_attn.unembed_out.scales": "model-00054.safetensors",
+    "model.layers.27.self_attn.unembed_out.biases": "model-00054.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.gate.e_score_correction_bias": "model-00055.safetensors",
+    "model.layers.28.mlp.gate.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.down_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.down_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.down_proj.biases": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.gate_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.gate_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.gate_proj.biases": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.up_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.up_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.shared_experts.up_proj.biases": "model-00055.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.kv_a_layernorm.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.kv_a_proj_with_mqa.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.kv_a_proj_with_mqa.scales": "model-00055.safetensors",
+    "model.layers.28.self_attn.kv_a_proj_with_mqa.biases": "model-00055.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.o_proj.scales": "model-00055.safetensors",
+    "model.layers.28.self_attn.o_proj.biases": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_a_layernorm.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_a_proj.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_a_proj.scales": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_a_proj.biases": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_b_proj.weight": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_b_proj.scales": "model-00055.safetensors",
+    "model.layers.28.self_attn.q_b_proj.biases": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.gate_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.gate_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.gate_proj.biases": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.down_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.down_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.down_proj.biases": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.up_proj.weight": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.up_proj.scales": "model-00055.safetensors",
+    "model.layers.28.mlp.switch_mlp.up_proj.biases": "model-00055.safetensors",
+    "model.layers.28.self_attn.embed_q.weight": "model-00056.safetensors",
+    "model.layers.28.self_attn.embed_q.scales": "model-00056.safetensors",
+    "model.layers.28.self_attn.embed_q.biases": "model-00056.safetensors",
+    "model.layers.28.self_attn.unembed_out.weight": "model-00056.safetensors",
+    "model.layers.28.self_attn.unembed_out.scales": "model-00056.safetensors",
+    "model.layers.28.self_attn.unembed_out.biases": "model-00056.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.gate.e_score_correction_bias": "model-00057.safetensors",
+    "model.layers.29.mlp.gate.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.down_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.down_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.down_proj.biases": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.gate_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.gate_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.gate_proj.biases": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.up_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.up_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.shared_experts.up_proj.biases": "model-00057.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.kv_a_layernorm.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.kv_a_proj_with_mqa.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.kv_a_proj_with_mqa.scales": "model-00057.safetensors",
+    "model.layers.29.self_attn.kv_a_proj_with_mqa.biases": "model-00057.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.o_proj.scales": "model-00057.safetensors",
+    "model.layers.29.self_attn.o_proj.biases": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_a_layernorm.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_a_proj.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_a_proj.scales": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_a_proj.biases": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_b_proj.weight": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_b_proj.scales": "model-00057.safetensors",
+    "model.layers.29.self_attn.q_b_proj.biases": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.gate_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.gate_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.gate_proj.biases": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.down_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.down_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.down_proj.biases": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.up_proj.weight": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.up_proj.scales": "model-00057.safetensors",
+    "model.layers.29.mlp.switch_mlp.up_proj.biases": "model-00057.safetensors",
+    "model.layers.29.self_attn.embed_q.weight": "model-00058.safetensors",
+    "model.layers.29.self_attn.embed_q.scales": "model-00058.safetensors",
+    "model.layers.29.self_attn.embed_q.biases": "model-00058.safetensors",
+    "model.layers.29.self_attn.unembed_out.weight": "model-00058.safetensors",
+    "model.layers.29.self_attn.unembed_out.scales": "model-00058.safetensors",
+    "model.layers.29.self_attn.unembed_out.biases": "model-00058.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.gate.e_score_correction_bias": "model-00059.safetensors",
+    "model.layers.30.mlp.gate.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.down_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.down_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.down_proj.biases": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.gate_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.gate_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.gate_proj.biases": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.up_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.up_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.shared_experts.up_proj.biases": "model-00059.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.indexer.k_norm.bias": "model-00059.safetensors",
+    "model.layers.30.self_attn.indexer.k_norm.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.indexer.weights_proj.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.indexer.wk.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.indexer.wq_b.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.kv_a_layernorm.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.kv_a_proj_with_mqa.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.kv_a_proj_with_mqa.scales": "model-00059.safetensors",
+    "model.layers.30.self_attn.kv_a_proj_with_mqa.biases": "model-00059.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.o_proj.scales": "model-00059.safetensors",
+    "model.layers.30.self_attn.o_proj.biases": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_a_layernorm.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_a_proj.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_a_proj.scales": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_a_proj.biases": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_b_proj.weight": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_b_proj.scales": "model-00059.safetensors",
+    "model.layers.30.self_attn.q_b_proj.biases": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.gate_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.gate_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.gate_proj.biases": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.down_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.down_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.down_proj.biases": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.up_proj.weight": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.up_proj.scales": "model-00059.safetensors",
+    "model.layers.30.mlp.switch_mlp.up_proj.biases": "model-00059.safetensors",
+    "model.layers.30.self_attn.embed_q.weight": "model-00060.safetensors",
+    "model.layers.30.self_attn.embed_q.scales": "model-00060.safetensors",
+    "model.layers.30.self_attn.embed_q.biases": "model-00060.safetensors",
+    "model.layers.30.self_attn.unembed_out.weight": "model-00060.safetensors",
+    "model.layers.30.self_attn.unembed_out.scales": "model-00060.safetensors",
+    "model.layers.30.self_attn.unembed_out.biases": "model-00060.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.gate.e_score_correction_bias": "model-00061.safetensors",
+    "model.layers.31.mlp.gate.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.down_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.down_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.down_proj.biases": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.gate_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.gate_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.gate_proj.biases": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.up_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.up_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.shared_experts.up_proj.biases": "model-00061.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.kv_a_layernorm.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.kv_a_proj_with_mqa.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.kv_a_proj_with_mqa.scales": "model-00061.safetensors",
+    "model.layers.31.self_attn.kv_a_proj_with_mqa.biases": "model-00061.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.o_proj.scales": "model-00061.safetensors",
+    "model.layers.31.self_attn.o_proj.biases": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_a_layernorm.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_a_proj.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_a_proj.scales": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_a_proj.biases": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_b_proj.weight": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_b_proj.scales": "model-00061.safetensors",
+    "model.layers.31.self_attn.q_b_proj.biases": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.gate_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.gate_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.gate_proj.biases": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.down_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.down_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.down_proj.biases": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.up_proj.weight": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.up_proj.scales": "model-00061.safetensors",
+    "model.layers.31.mlp.switch_mlp.up_proj.biases": "model-00061.safetensors",
+    "model.layers.31.self_attn.embed_q.weight": "model-00062.safetensors",
+    "model.layers.31.self_attn.embed_q.scales": "model-00062.safetensors",
+    "model.layers.31.self_attn.embed_q.biases": "model-00062.safetensors",
+    "model.layers.31.self_attn.unembed_out.weight": "model-00062.safetensors",
+    "model.layers.31.self_attn.unembed_out.scales": "model-00062.safetensors",
+    "model.layers.31.self_attn.unembed_out.biases": "model-00062.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.gate.e_score_correction_bias": "model-00063.safetensors",
+    "model.layers.32.mlp.gate.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.down_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.down_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.down_proj.biases": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.gate_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.gate_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.gate_proj.biases": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.up_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.up_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.shared_experts.up_proj.biases": "model-00063.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.kv_a_layernorm.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.kv_a_proj_with_mqa.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.kv_a_proj_with_mqa.scales": "model-00063.safetensors",
+    "model.layers.32.self_attn.kv_a_proj_with_mqa.biases": "model-00063.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.o_proj.scales": "model-00063.safetensors",
+    "model.layers.32.self_attn.o_proj.biases": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_a_layernorm.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_a_proj.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_a_proj.scales": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_a_proj.biases": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_b_proj.weight": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_b_proj.scales": "model-00063.safetensors",
+    "model.layers.32.self_attn.q_b_proj.biases": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.gate_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.gate_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.gate_proj.biases": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.down_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.down_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.down_proj.biases": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.up_proj.weight": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.up_proj.scales": "model-00063.safetensors",
+    "model.layers.32.mlp.switch_mlp.up_proj.biases": "model-00063.safetensors",
+    "model.layers.32.self_attn.embed_q.weight": "model-00064.safetensors",
+    "model.layers.32.self_attn.embed_q.scales": "model-00064.safetensors",
+    "model.layers.32.self_attn.embed_q.biases": "model-00064.safetensors",
+    "model.layers.32.self_attn.unembed_out.weight": "model-00064.safetensors",
+    "model.layers.32.self_attn.unembed_out.scales": "model-00064.safetensors",
+    "model.layers.32.self_attn.unembed_out.biases": "model-00064.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.gate.e_score_correction_bias": "model-00065.safetensors",
+    "model.layers.33.mlp.gate.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.down_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.down_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.down_proj.biases": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.gate_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.gate_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.gate_proj.biases": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.up_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.up_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.shared_experts.up_proj.biases": "model-00065.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.kv_a_layernorm.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.kv_a_proj_with_mqa.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.kv_a_proj_with_mqa.scales": "model-00065.safetensors",
+    "model.layers.33.self_attn.kv_a_proj_with_mqa.biases": "model-00065.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.o_proj.scales": "model-00065.safetensors",
+    "model.layers.33.self_attn.o_proj.biases": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_a_layernorm.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_a_proj.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_a_proj.scales": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_a_proj.biases": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_b_proj.weight": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_b_proj.scales": "model-00065.safetensors",
+    "model.layers.33.self_attn.q_b_proj.biases": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.gate_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.gate_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.gate_proj.biases": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.down_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.down_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.down_proj.biases": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.up_proj.weight": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.up_proj.scales": "model-00065.safetensors",
+    "model.layers.33.mlp.switch_mlp.up_proj.biases": "model-00065.safetensors",
+    "model.layers.33.self_attn.embed_q.weight": "model-00066.safetensors",
+    "model.layers.33.self_attn.embed_q.scales": "model-00066.safetensors",
+    "model.layers.33.self_attn.embed_q.biases": "model-00066.safetensors",
+    "model.layers.33.self_attn.unembed_out.weight": "model-00066.safetensors",
+    "model.layers.33.self_attn.unembed_out.scales": "model-00066.safetensors",
+    "model.layers.33.self_attn.unembed_out.biases": "model-00066.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.gate.e_score_correction_bias": "model-00067.safetensors",
+    "model.layers.34.mlp.gate.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.down_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.down_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.down_proj.biases": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.gate_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.gate_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.gate_proj.biases": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.up_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.up_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.shared_experts.up_proj.biases": "model-00067.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.indexer.k_norm.bias": "model-00067.safetensors",
+    "model.layers.34.self_attn.indexer.k_norm.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.indexer.weights_proj.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.indexer.wk.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.indexer.wq_b.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.kv_a_layernorm.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.kv_a_proj_with_mqa.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.kv_a_proj_with_mqa.scales": "model-00067.safetensors",
+    "model.layers.34.self_attn.kv_a_proj_with_mqa.biases": "model-00067.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.o_proj.scales": "model-00067.safetensors",
+    "model.layers.34.self_attn.o_proj.biases": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_a_layernorm.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_a_proj.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_a_proj.scales": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_a_proj.biases": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_b_proj.weight": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_b_proj.scales": "model-00067.safetensors",
+    "model.layers.34.self_attn.q_b_proj.biases": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.gate_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.gate_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.gate_proj.biases": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.down_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.down_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.down_proj.biases": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.up_proj.weight": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.up_proj.scales": "model-00067.safetensors",
+    "model.layers.34.mlp.switch_mlp.up_proj.biases": "model-00067.safetensors",
+    "model.layers.34.self_attn.embed_q.weight": "model-00068.safetensors",
+    "model.layers.34.self_attn.embed_q.scales": "model-00068.safetensors",
+    "model.layers.34.self_attn.embed_q.biases": "model-00068.safetensors",
+    "model.layers.34.self_attn.unembed_out.weight": "model-00068.safetensors",
+    "model.layers.34.self_attn.unembed_out.scales": "model-00068.safetensors",
+    "model.layers.34.self_attn.unembed_out.biases": "model-00068.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.gate.e_score_correction_bias": "model-00069.safetensors",
+    "model.layers.35.mlp.gate.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.down_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.down_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.down_proj.biases": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.gate_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.gate_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.gate_proj.biases": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.up_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.up_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.shared_experts.up_proj.biases": "model-00069.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.kv_a_layernorm.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.kv_a_proj_with_mqa.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.kv_a_proj_with_mqa.scales": "model-00069.safetensors",
+    "model.layers.35.self_attn.kv_a_proj_with_mqa.biases": "model-00069.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.o_proj.scales": "model-00069.safetensors",
+    "model.layers.35.self_attn.o_proj.biases": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_a_layernorm.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_a_proj.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_a_proj.scales": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_a_proj.biases": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_b_proj.weight": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_b_proj.scales": "model-00069.safetensors",
+    "model.layers.35.self_attn.q_b_proj.biases": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.gate_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.gate_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.gate_proj.biases": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.down_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.down_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.down_proj.biases": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.up_proj.weight": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.up_proj.scales": "model-00069.safetensors",
+    "model.layers.35.mlp.switch_mlp.up_proj.biases": "model-00069.safetensors",
+    "model.layers.35.self_attn.embed_q.weight": "model-00070.safetensors",
+    "model.layers.35.self_attn.embed_q.scales": "model-00070.safetensors",
+    "model.layers.35.self_attn.embed_q.biases": "model-00070.safetensors",
+    "model.layers.35.self_attn.unembed_out.weight": "model-00070.safetensors",
+    "model.layers.35.self_attn.unembed_out.scales": "model-00070.safetensors",
+    "model.layers.35.self_attn.unembed_out.biases": "model-00070.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.gate.e_score_correction_bias": "model-00071.safetensors",
+    "model.layers.36.mlp.gate.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.down_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.down_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.down_proj.biases": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.gate_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.gate_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.gate_proj.biases": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.up_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.up_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.shared_experts.up_proj.biases": "model-00071.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.kv_a_layernorm.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.kv_a_proj_with_mqa.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.kv_a_proj_with_mqa.scales": "model-00071.safetensors",
+    "model.layers.36.self_attn.kv_a_proj_with_mqa.biases": "model-00071.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.o_proj.scales": "model-00071.safetensors",
+    "model.layers.36.self_attn.o_proj.biases": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_a_layernorm.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_a_proj.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_a_proj.scales": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_a_proj.biases": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_b_proj.weight": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_b_proj.scales": "model-00071.safetensors",
+    "model.layers.36.self_attn.q_b_proj.biases": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.gate_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.gate_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.gate_proj.biases": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.down_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.down_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.down_proj.biases": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.up_proj.weight": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.up_proj.scales": "model-00071.safetensors",
+    "model.layers.36.mlp.switch_mlp.up_proj.biases": "model-00071.safetensors",
+    "model.layers.36.self_attn.embed_q.weight": "model-00072.safetensors",
+    "model.layers.36.self_attn.embed_q.scales": "model-00072.safetensors",
+    "model.layers.36.self_attn.embed_q.biases": "model-00072.safetensors",
+    "model.layers.36.self_attn.unembed_out.weight": "model-00072.safetensors",
+    "model.layers.36.self_attn.unembed_out.scales": "model-00072.safetensors",
+    "model.layers.36.self_attn.unembed_out.biases": "model-00072.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.gate.e_score_correction_bias": "model-00073.safetensors",
+    "model.layers.37.mlp.gate.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.down_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.down_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.down_proj.biases": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.gate_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.gate_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.gate_proj.biases": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.up_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.up_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.shared_experts.up_proj.biases": "model-00073.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.kv_a_layernorm.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.kv_a_proj_with_mqa.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.kv_a_proj_with_mqa.scales": "model-00073.safetensors",
+    "model.layers.37.self_attn.kv_a_proj_with_mqa.biases": "model-00073.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.o_proj.scales": "model-00073.safetensors",
+    "model.layers.37.self_attn.o_proj.biases": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_a_layernorm.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_a_proj.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_a_proj.scales": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_a_proj.biases": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_b_proj.weight": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_b_proj.scales": "model-00073.safetensors",
+    "model.layers.37.self_attn.q_b_proj.biases": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.gate_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.gate_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.gate_proj.biases": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.down_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.down_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.down_proj.biases": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.up_proj.weight": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.up_proj.scales": "model-00073.safetensors",
+    "model.layers.37.mlp.switch_mlp.up_proj.biases": "model-00073.safetensors",
+    "model.layers.37.self_attn.embed_q.weight": "model-00074.safetensors",
+    "model.layers.37.self_attn.embed_q.scales": "model-00074.safetensors",
+    "model.layers.37.self_attn.embed_q.biases": "model-00074.safetensors",
+    "model.layers.37.self_attn.unembed_out.weight": "model-00074.safetensors",
+    "model.layers.37.self_attn.unembed_out.scales": "model-00074.safetensors",
+    "model.layers.37.self_attn.unembed_out.biases": "model-00074.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.gate.e_score_correction_bias": "model-00075.safetensors",
+    "model.layers.38.mlp.gate.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.down_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.down_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.down_proj.biases": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.gate_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.gate_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.gate_proj.biases": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.up_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.up_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.shared_experts.up_proj.biases": "model-00075.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.indexer.k_norm.bias": "model-00075.safetensors",
+    "model.layers.38.self_attn.indexer.k_norm.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.indexer.weights_proj.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.indexer.wk.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.indexer.wq_b.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.kv_a_layernorm.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.kv_a_proj_with_mqa.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.kv_a_proj_with_mqa.scales": "model-00075.safetensors",
+    "model.layers.38.self_attn.kv_a_proj_with_mqa.biases": "model-00075.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.o_proj.scales": "model-00075.safetensors",
+    "model.layers.38.self_attn.o_proj.biases": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_a_layernorm.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_a_proj.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_a_proj.scales": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_a_proj.biases": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_b_proj.weight": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_b_proj.scales": "model-00075.safetensors",
+    "model.layers.38.self_attn.q_b_proj.biases": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.gate_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.gate_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.gate_proj.biases": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.down_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.down_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.down_proj.biases": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.up_proj.weight": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.up_proj.scales": "model-00075.safetensors",
+    "model.layers.38.mlp.switch_mlp.up_proj.biases": "model-00075.safetensors",
+    "model.layers.38.self_attn.embed_q.weight": "model-00076.safetensors",
+    "model.layers.38.self_attn.embed_q.scales": "model-00076.safetensors",
+    "model.layers.38.self_attn.embed_q.biases": "model-00076.safetensors",
+    "model.layers.38.self_attn.unembed_out.weight": "model-00076.safetensors",
+    "model.layers.38.self_attn.unembed_out.scales": "model-00076.safetensors",
+    "model.layers.38.self_attn.unembed_out.biases": "model-00076.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.gate.e_score_correction_bias": "model-00077.safetensors",
+    "model.layers.39.mlp.gate.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.down_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.down_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.down_proj.biases": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.gate_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.gate_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.gate_proj.biases": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.up_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.up_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.shared_experts.up_proj.biases": "model-00077.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.kv_a_layernorm.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.kv_a_proj_with_mqa.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.kv_a_proj_with_mqa.scales": "model-00077.safetensors",
+    "model.layers.39.self_attn.kv_a_proj_with_mqa.biases": "model-00077.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.o_proj.scales": "model-00077.safetensors",
+    "model.layers.39.self_attn.o_proj.biases": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_a_layernorm.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_a_proj.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_a_proj.scales": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_a_proj.biases": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_b_proj.weight": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_b_proj.scales": "model-00077.safetensors",
+    "model.layers.39.self_attn.q_b_proj.biases": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.gate_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.gate_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.gate_proj.biases": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.down_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.down_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.down_proj.biases": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.up_proj.weight": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.up_proj.scales": "model-00077.safetensors",
+    "model.layers.39.mlp.switch_mlp.up_proj.biases": "model-00077.safetensors",
+    "model.layers.39.self_attn.embed_q.weight": "model-00078.safetensors",
+    "model.layers.39.self_attn.embed_q.scales": "model-00078.safetensors",
+    "model.layers.39.self_attn.embed_q.biases": "model-00078.safetensors",
+    "model.layers.39.self_attn.unembed_out.weight": "model-00078.safetensors",
+    "model.layers.39.self_attn.unembed_out.scales": "model-00078.safetensors",
+    "model.layers.39.self_attn.unembed_out.biases": "model-00078.safetensors",
+    "model.layers.40.input_layernorm.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.gate.e_score_correction_bias": "model-00079.safetensors",
+    "model.layers.40.mlp.gate.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.down_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.down_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.down_proj.biases": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.gate_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.gate_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.gate_proj.biases": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.up_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.up_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.shared_experts.up_proj.biases": "model-00079.safetensors",
+    "model.layers.40.post_attention_layernorm.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.kv_a_layernorm.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.kv_a_proj_with_mqa.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.kv_a_proj_with_mqa.scales": "model-00079.safetensors",
+    "model.layers.40.self_attn.kv_a_proj_with_mqa.biases": "model-00079.safetensors",
+    "model.layers.40.self_attn.o_proj.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.o_proj.scales": "model-00079.safetensors",
+    "model.layers.40.self_attn.o_proj.biases": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_a_layernorm.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_a_proj.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_a_proj.scales": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_a_proj.biases": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_b_proj.weight": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_b_proj.scales": "model-00079.safetensors",
+    "model.layers.40.self_attn.q_b_proj.biases": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.gate_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.gate_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.gate_proj.biases": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.down_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.down_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.down_proj.biases": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.up_proj.weight": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.up_proj.scales": "model-00079.safetensors",
+    "model.layers.40.mlp.switch_mlp.up_proj.biases": "model-00079.safetensors",
+    "model.layers.40.self_attn.embed_q.weight": "model-00080.safetensors",
+    "model.layers.40.self_attn.embed_q.scales": "model-00080.safetensors",
+    "model.layers.40.self_attn.embed_q.biases": "model-00080.safetensors",
+    "model.layers.40.self_attn.unembed_out.weight": "model-00080.safetensors",
+    "model.layers.40.self_attn.unembed_out.scales": "model-00080.safetensors",
+    "model.layers.40.self_attn.unembed_out.biases": "model-00080.safetensors",
+    "model.layers.41.input_layernorm.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.gate.e_score_correction_bias": "model-00081.safetensors",
+    "model.layers.41.mlp.gate.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.down_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.down_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.down_proj.biases": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.gate_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.gate_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.gate_proj.biases": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.up_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.up_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.shared_experts.up_proj.biases": "model-00081.safetensors",
+    "model.layers.41.post_attention_layernorm.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.kv_a_layernorm.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.kv_a_proj_with_mqa.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.kv_a_proj_with_mqa.scales": "model-00081.safetensors",
+    "model.layers.41.self_attn.kv_a_proj_with_mqa.biases": "model-00081.safetensors",
+    "model.layers.41.self_attn.o_proj.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.o_proj.scales": "model-00081.safetensors",
+    "model.layers.41.self_attn.o_proj.biases": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_a_layernorm.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_a_proj.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_a_proj.scales": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_a_proj.biases": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_b_proj.weight": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_b_proj.scales": "model-00081.safetensors",
+    "model.layers.41.self_attn.q_b_proj.biases": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.gate_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.gate_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.gate_proj.biases": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.down_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.down_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.down_proj.biases": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.up_proj.weight": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.up_proj.scales": "model-00081.safetensors",
+    "model.layers.41.mlp.switch_mlp.up_proj.biases": "model-00081.safetensors",
+    "model.layers.41.self_attn.embed_q.weight": "model-00082.safetensors",
+    "model.layers.41.self_attn.embed_q.scales": "model-00082.safetensors",
+    "model.layers.41.self_attn.embed_q.biases": "model-00082.safetensors",
+    "model.layers.41.self_attn.unembed_out.weight": "model-00082.safetensors",
+    "model.layers.41.self_attn.unembed_out.scales": "model-00082.safetensors",
+    "model.layers.41.self_attn.unembed_out.biases": "model-00082.safetensors",
+    "model.layers.42.input_layernorm.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.gate.e_score_correction_bias": "model-00083.safetensors",
+    "model.layers.42.mlp.gate.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.down_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.down_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.down_proj.biases": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.gate_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.gate_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.gate_proj.biases": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.up_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.up_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.shared_experts.up_proj.biases": "model-00083.safetensors",
+    "model.layers.42.post_attention_layernorm.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.indexer.k_norm.bias": "model-00083.safetensors",
+    "model.layers.42.self_attn.indexer.k_norm.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.indexer.weights_proj.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.indexer.wk.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.indexer.wq_b.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.kv_a_layernorm.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.kv_a_proj_with_mqa.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.kv_a_proj_with_mqa.scales": "model-00083.safetensors",
+    "model.layers.42.self_attn.kv_a_proj_with_mqa.biases": "model-00083.safetensors",
+    "model.layers.42.self_attn.o_proj.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.o_proj.scales": "model-00083.safetensors",
+    "model.layers.42.self_attn.o_proj.biases": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_a_layernorm.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_a_proj.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_a_proj.scales": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_a_proj.biases": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_b_proj.weight": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_b_proj.scales": "model-00083.safetensors",
+    "model.layers.42.self_attn.q_b_proj.biases": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.gate_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.gate_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.gate_proj.biases": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.down_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.down_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.down_proj.biases": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.up_proj.weight": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.up_proj.scales": "model-00083.safetensors",
+    "model.layers.42.mlp.switch_mlp.up_proj.biases": "model-00083.safetensors",
+    "model.layers.42.self_attn.embed_q.weight": "model-00084.safetensors",
+    "model.layers.42.self_attn.embed_q.scales": "model-00084.safetensors",
+    "model.layers.42.self_attn.embed_q.biases": "model-00084.safetensors",
+    "model.layers.42.self_attn.unembed_out.weight": "model-00084.safetensors",
+    "model.layers.42.self_attn.unembed_out.scales": "model-00084.safetensors",
+    "model.layers.42.self_attn.unembed_out.biases": "model-00084.safetensors",
+    "model.layers.43.input_layernorm.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.gate.e_score_correction_bias": "model-00085.safetensors",
+    "model.layers.43.mlp.gate.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.down_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.down_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.down_proj.biases": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.gate_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.gate_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.gate_proj.biases": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.up_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.up_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.shared_experts.up_proj.biases": "model-00085.safetensors",
+    "model.layers.43.post_attention_layernorm.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.kv_a_layernorm.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.kv_a_proj_with_mqa.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.kv_a_proj_with_mqa.scales": "model-00085.safetensors",
+    "model.layers.43.self_attn.kv_a_proj_with_mqa.biases": "model-00085.safetensors",
+    "model.layers.43.self_attn.o_proj.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.o_proj.scales": "model-00085.safetensors",
+    "model.layers.43.self_attn.o_proj.biases": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_a_layernorm.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_a_proj.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_a_proj.scales": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_a_proj.biases": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_b_proj.weight": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_b_proj.scales": "model-00085.safetensors",
+    "model.layers.43.self_attn.q_b_proj.biases": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.gate_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.gate_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.gate_proj.biases": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.down_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.down_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.down_proj.biases": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.up_proj.weight": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.up_proj.scales": "model-00085.safetensors",
+    "model.layers.43.mlp.switch_mlp.up_proj.biases": "model-00085.safetensors",
+    "model.layers.43.self_attn.embed_q.weight": "model-00086.safetensors",
+    "model.layers.43.self_attn.embed_q.scales": "model-00086.safetensors",
+    "model.layers.43.self_attn.embed_q.biases": "model-00086.safetensors",
+    "model.layers.43.self_attn.unembed_out.weight": "model-00086.safetensors",
+    "model.layers.43.self_attn.unembed_out.scales": "model-00086.safetensors",
+    "model.layers.43.self_attn.unembed_out.biases": "model-00086.safetensors",
+    "model.layers.44.input_layernorm.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.gate.e_score_correction_bias": "model-00087.safetensors",
+    "model.layers.44.mlp.gate.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.down_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.down_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.down_proj.biases": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.gate_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.gate_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.gate_proj.biases": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.up_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.up_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.shared_experts.up_proj.biases": "model-00087.safetensors",
+    "model.layers.44.post_attention_layernorm.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.kv_a_layernorm.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.kv_a_proj_with_mqa.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.kv_a_proj_with_mqa.scales": "model-00087.safetensors",
+    "model.layers.44.self_attn.kv_a_proj_with_mqa.biases": "model-00087.safetensors",
+    "model.layers.44.self_attn.o_proj.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.o_proj.scales": "model-00087.safetensors",
+    "model.layers.44.self_attn.o_proj.biases": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_a_layernorm.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_a_proj.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_a_proj.scales": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_a_proj.biases": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_b_proj.weight": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_b_proj.scales": "model-00087.safetensors",
+    "model.layers.44.self_attn.q_b_proj.biases": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.gate_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.gate_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.gate_proj.biases": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.down_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.down_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.down_proj.biases": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.up_proj.weight": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.up_proj.scales": "model-00087.safetensors",
+    "model.layers.44.mlp.switch_mlp.up_proj.biases": "model-00087.safetensors",
+    "model.layers.44.self_attn.embed_q.weight": "model-00088.safetensors",
+    "model.layers.44.self_attn.embed_q.scales": "model-00088.safetensors",
+    "model.layers.44.self_attn.embed_q.biases": "model-00088.safetensors",
+    "model.layers.44.self_attn.unembed_out.weight": "model-00088.safetensors",
+    "model.layers.44.self_attn.unembed_out.scales": "model-00088.safetensors",
+    "model.layers.44.self_attn.unembed_out.biases": "model-00088.safetensors",
+    "model.layers.45.input_layernorm.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.gate.e_score_correction_bias": "model-00089.safetensors",
+    "model.layers.45.mlp.gate.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.down_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.down_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.down_proj.biases": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.gate_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.gate_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.gate_proj.biases": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.up_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.up_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.shared_experts.up_proj.biases": "model-00089.safetensors",
+    "model.layers.45.post_attention_layernorm.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.kv_a_layernorm.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.kv_a_proj_with_mqa.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.kv_a_proj_with_mqa.scales": "model-00089.safetensors",
+    "model.layers.45.self_attn.kv_a_proj_with_mqa.biases": "model-00089.safetensors",
+    "model.layers.45.self_attn.o_proj.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.o_proj.scales": "model-00089.safetensors",
+    "model.layers.45.self_attn.o_proj.biases": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_a_layernorm.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_a_proj.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_a_proj.scales": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_a_proj.biases": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_b_proj.weight": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_b_proj.scales": "model-00089.safetensors",
+    "model.layers.45.self_attn.q_b_proj.biases": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.gate_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.gate_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.gate_proj.biases": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.down_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.down_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.down_proj.biases": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.up_proj.weight": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.up_proj.scales": "model-00089.safetensors",
+    "model.layers.45.mlp.switch_mlp.up_proj.biases": "model-00089.safetensors",
+    "model.layers.45.self_attn.embed_q.weight": "model-00090.safetensors",
+    "model.layers.45.self_attn.embed_q.scales": "model-00090.safetensors",
+    "model.layers.45.self_attn.embed_q.biases": "model-00090.safetensors",
+    "model.layers.45.self_attn.unembed_out.weight": "model-00090.safetensors",
+    "model.layers.45.self_attn.unembed_out.scales": "model-00090.safetensors",
+    "model.layers.45.self_attn.unembed_out.biases": "model-00090.safetensors",
+    "model.layers.46.input_layernorm.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.gate.e_score_correction_bias": "model-00091.safetensors",
+    "model.layers.46.mlp.gate.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.down_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.down_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.down_proj.biases": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.gate_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.gate_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.gate_proj.biases": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.up_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.up_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.shared_experts.up_proj.biases": "model-00091.safetensors",
+    "model.layers.46.post_attention_layernorm.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.indexer.k_norm.bias": "model-00091.safetensors",
+    "model.layers.46.self_attn.indexer.k_norm.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.indexer.weights_proj.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.indexer.wk.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.indexer.wq_b.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.kv_a_layernorm.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.kv_a_proj_with_mqa.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.kv_a_proj_with_mqa.scales": "model-00091.safetensors",
+    "model.layers.46.self_attn.kv_a_proj_with_mqa.biases": "model-00091.safetensors",
+    "model.layers.46.self_attn.o_proj.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.o_proj.scales": "model-00091.safetensors",
+    "model.layers.46.self_attn.o_proj.biases": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_a_layernorm.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_a_proj.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_a_proj.scales": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_a_proj.biases": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_b_proj.weight": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_b_proj.scales": "model-00091.safetensors",
+    "model.layers.46.self_attn.q_b_proj.biases": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.gate_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.gate_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.gate_proj.biases": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.down_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.down_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.down_proj.biases": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.up_proj.weight": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.up_proj.scales": "model-00091.safetensors",
+    "model.layers.46.mlp.switch_mlp.up_proj.biases": "model-00091.safetensors",
+    "model.layers.46.self_attn.embed_q.weight": "model-00092.safetensors",
+    "model.layers.46.self_attn.embed_q.scales": "model-00092.safetensors",
+    "model.layers.46.self_attn.embed_q.biases": "model-00092.safetensors",
+    "model.layers.46.self_attn.unembed_out.weight": "model-00092.safetensors",
+    "model.layers.46.self_attn.unembed_out.scales": "model-00092.safetensors",
+    "model.layers.46.self_attn.unembed_out.biases": "model-00092.safetensors",
+    "model.layers.47.input_layernorm.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.gate.e_score_correction_bias": "model-00093.safetensors",
+    "model.layers.47.mlp.gate.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.down_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.down_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.down_proj.biases": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.gate_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.gate_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.gate_proj.biases": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.up_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.up_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.shared_experts.up_proj.biases": "model-00093.safetensors",
+    "model.layers.47.post_attention_layernorm.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.kv_a_layernorm.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.kv_a_proj_with_mqa.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.kv_a_proj_with_mqa.scales": "model-00093.safetensors",
+    "model.layers.47.self_attn.kv_a_proj_with_mqa.biases": "model-00093.safetensors",
+    "model.layers.47.self_attn.o_proj.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.o_proj.scales": "model-00093.safetensors",
+    "model.layers.47.self_attn.o_proj.biases": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_a_layernorm.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_a_proj.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_a_proj.scales": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_a_proj.biases": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_b_proj.weight": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_b_proj.scales": "model-00093.safetensors",
+    "model.layers.47.self_attn.q_b_proj.biases": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.gate_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.gate_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.gate_proj.biases": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.down_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.down_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.down_proj.biases": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.up_proj.weight": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.up_proj.scales": "model-00093.safetensors",
+    "model.layers.47.mlp.switch_mlp.up_proj.biases": "model-00093.safetensors",
+    "model.layers.47.self_attn.embed_q.weight": "model-00094.safetensors",
+    "model.layers.47.self_attn.embed_q.scales": "model-00094.safetensors",
+    "model.layers.47.self_attn.embed_q.biases": "model-00094.safetensors",
+    "model.layers.47.self_attn.unembed_out.weight": "model-00094.safetensors",
+    "model.layers.47.self_attn.unembed_out.scales": "model-00094.safetensors",
+    "model.layers.47.self_attn.unembed_out.biases": "model-00094.safetensors",
+    "model.layers.48.input_layernorm.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.gate.e_score_correction_bias": "model-00095.safetensors",
+    "model.layers.48.mlp.gate.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.down_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.down_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.down_proj.biases": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.gate_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.gate_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.gate_proj.biases": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.up_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.up_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.shared_experts.up_proj.biases": "model-00095.safetensors",
+    "model.layers.48.post_attention_layernorm.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.kv_a_layernorm.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.kv_a_proj_with_mqa.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.kv_a_proj_with_mqa.scales": "model-00095.safetensors",
+    "model.layers.48.self_attn.kv_a_proj_with_mqa.biases": "model-00095.safetensors",
+    "model.layers.48.self_attn.o_proj.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.o_proj.scales": "model-00095.safetensors",
+    "model.layers.48.self_attn.o_proj.biases": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_a_layernorm.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_a_proj.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_a_proj.scales": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_a_proj.biases": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_b_proj.weight": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_b_proj.scales": "model-00095.safetensors",
+    "model.layers.48.self_attn.q_b_proj.biases": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.gate_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.gate_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.gate_proj.biases": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.down_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.down_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.down_proj.biases": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.up_proj.weight": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.up_proj.scales": "model-00095.safetensors",
+    "model.layers.48.mlp.switch_mlp.up_proj.biases": "model-00095.safetensors",
+    "model.layers.48.self_attn.embed_q.weight": "model-00096.safetensors",
+    "model.layers.48.self_attn.embed_q.scales": "model-00096.safetensors",
+    "model.layers.48.self_attn.embed_q.biases": "model-00096.safetensors",
+    "model.layers.48.self_attn.unembed_out.weight": "model-00096.safetensors",
+    "model.layers.48.self_attn.unembed_out.scales": "model-00096.safetensors",
+    "model.layers.48.self_attn.unembed_out.biases": "model-00096.safetensors",
+    "model.layers.49.input_layernorm.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.gate.e_score_correction_bias": "model-00097.safetensors",
+    "model.layers.49.mlp.gate.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.down_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.down_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.down_proj.biases": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.gate_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.gate_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.gate_proj.biases": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.up_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.up_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.shared_experts.up_proj.biases": "model-00097.safetensors",
+    "model.layers.49.post_attention_layernorm.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.kv_a_layernorm.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.kv_a_proj_with_mqa.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.kv_a_proj_with_mqa.scales": "model-00097.safetensors",
+    "model.layers.49.self_attn.kv_a_proj_with_mqa.biases": "model-00097.safetensors",
+    "model.layers.49.self_attn.o_proj.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.o_proj.scales": "model-00097.safetensors",
+    "model.layers.49.self_attn.o_proj.biases": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_a_layernorm.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_a_proj.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_a_proj.scales": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_a_proj.biases": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_b_proj.weight": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_b_proj.scales": "model-00097.safetensors",
+    "model.layers.49.self_attn.q_b_proj.biases": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.gate_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.gate_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.gate_proj.biases": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.down_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.down_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.down_proj.biases": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.up_proj.weight": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.up_proj.scales": "model-00097.safetensors",
+    "model.layers.49.mlp.switch_mlp.up_proj.biases": "model-00097.safetensors",
+    "model.layers.49.self_attn.embed_q.weight": "model-00098.safetensors",
+    "model.layers.49.self_attn.embed_q.scales": "model-00098.safetensors",
+    "model.layers.49.self_attn.embed_q.biases": "model-00098.safetensors",
+    "model.layers.49.self_attn.unembed_out.weight": "model-00098.safetensors",
+    "model.layers.49.self_attn.unembed_out.scales": "model-00098.safetensors",
+    "model.layers.49.self_attn.unembed_out.biases": "model-00098.safetensors",
+    "model.layers.50.input_layernorm.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.gate.e_score_correction_bias": "model-00099.safetensors",
+    "model.layers.50.mlp.gate.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.down_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.down_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.down_proj.biases": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.gate_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.gate_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.gate_proj.biases": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.up_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.up_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.shared_experts.up_proj.biases": "model-00099.safetensors",
+    "model.layers.50.post_attention_layernorm.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.indexer.k_norm.bias": "model-00099.safetensors",
+    "model.layers.50.self_attn.indexer.k_norm.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.indexer.weights_proj.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.indexer.wk.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.indexer.wq_b.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.kv_a_layernorm.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.kv_a_proj_with_mqa.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.kv_a_proj_with_mqa.scales": "model-00099.safetensors",
+    "model.layers.50.self_attn.kv_a_proj_with_mqa.biases": "model-00099.safetensors",
+    "model.layers.50.self_attn.o_proj.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.o_proj.scales": "model-00099.safetensors",
+    "model.layers.50.self_attn.o_proj.biases": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_a_layernorm.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_a_proj.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_a_proj.scales": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_a_proj.biases": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_b_proj.weight": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_b_proj.scales": "model-00099.safetensors",
+    "model.layers.50.self_attn.q_b_proj.biases": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.gate_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.gate_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.gate_proj.biases": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.down_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.down_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.down_proj.biases": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.up_proj.weight": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.up_proj.scales": "model-00099.safetensors",
+    "model.layers.50.mlp.switch_mlp.up_proj.biases": "model-00099.safetensors",
+    "model.layers.50.self_attn.embed_q.weight": "model-00100.safetensors",
+    "model.layers.50.self_attn.embed_q.scales": "model-00100.safetensors",
+    "model.layers.50.self_attn.embed_q.biases": "model-00100.safetensors",
+    "model.layers.50.self_attn.unembed_out.weight": "model-00100.safetensors",
+    "model.layers.50.self_attn.unembed_out.scales": "model-00100.safetensors",
+    "model.layers.50.self_attn.unembed_out.biases": "model-00100.safetensors",
+    "model.layers.51.input_layernorm.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.gate.e_score_correction_bias": "model-00101.safetensors",
+    "model.layers.51.mlp.gate.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.down_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.down_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.down_proj.biases": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.gate_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.gate_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.gate_proj.biases": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.up_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.up_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.shared_experts.up_proj.biases": "model-00101.safetensors",
+    "model.layers.51.post_attention_layernorm.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.kv_a_layernorm.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.kv_a_proj_with_mqa.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.kv_a_proj_with_mqa.scales": "model-00101.safetensors",
+    "model.layers.51.self_attn.kv_a_proj_with_mqa.biases": "model-00101.safetensors",
+    "model.layers.51.self_attn.o_proj.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.o_proj.scales": "model-00101.safetensors",
+    "model.layers.51.self_attn.o_proj.biases": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_a_layernorm.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_a_proj.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_a_proj.scales": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_a_proj.biases": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_b_proj.weight": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_b_proj.scales": "model-00101.safetensors",
+    "model.layers.51.self_attn.q_b_proj.biases": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.gate_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.gate_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.gate_proj.biases": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.down_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.down_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.down_proj.biases": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.up_proj.weight": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.up_proj.scales": "model-00101.safetensors",
+    "model.layers.51.mlp.switch_mlp.up_proj.biases": "model-00101.safetensors",
+    "model.layers.51.self_attn.embed_q.weight": "model-00102.safetensors",
+    "model.layers.51.self_attn.embed_q.scales": "model-00102.safetensors",
+    "model.layers.51.self_attn.embed_q.biases": "model-00102.safetensors",
+    "model.layers.51.self_attn.unembed_out.weight": "model-00102.safetensors",
+    "model.layers.51.self_attn.unembed_out.scales": "model-00102.safetensors",
+    "model.layers.51.self_attn.unembed_out.biases": "model-00102.safetensors",
+    "model.layers.52.input_layernorm.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.gate.e_score_correction_bias": "model-00103.safetensors",
+    "model.layers.52.mlp.gate.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.down_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.down_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.down_proj.biases": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.gate_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.gate_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.gate_proj.biases": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.up_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.up_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.shared_experts.up_proj.biases": "model-00103.safetensors",
+    "model.layers.52.post_attention_layernorm.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.kv_a_layernorm.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.kv_a_proj_with_mqa.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.kv_a_proj_with_mqa.scales": "model-00103.safetensors",
+    "model.layers.52.self_attn.kv_a_proj_with_mqa.biases": "model-00103.safetensors",
+    "model.layers.52.self_attn.o_proj.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.o_proj.scales": "model-00103.safetensors",
+    "model.layers.52.self_attn.o_proj.biases": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_a_layernorm.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_a_proj.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_a_proj.scales": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_a_proj.biases": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_b_proj.weight": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_b_proj.scales": "model-00103.safetensors",
+    "model.layers.52.self_attn.q_b_proj.biases": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.gate_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.gate_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.gate_proj.biases": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.down_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.down_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.down_proj.biases": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.up_proj.weight": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.up_proj.scales": "model-00103.safetensors",
+    "model.layers.52.mlp.switch_mlp.up_proj.biases": "model-00103.safetensors",
+    "model.layers.52.self_attn.embed_q.weight": "model-00104.safetensors",
+    "model.layers.52.self_attn.embed_q.scales": "model-00104.safetensors",
+    "model.layers.52.self_attn.embed_q.biases": "model-00104.safetensors",
+    "model.layers.52.self_attn.unembed_out.weight": "model-00104.safetensors",
+    "model.layers.52.self_attn.unembed_out.scales": "model-00104.safetensors",
+    "model.layers.52.self_attn.unembed_out.biases": "model-00104.safetensors",
+    "model.layers.53.input_layernorm.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.gate.e_score_correction_bias": "model-00105.safetensors",
+    "model.layers.53.mlp.gate.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.down_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.down_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.down_proj.biases": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.gate_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.gate_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.gate_proj.biases": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.up_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.up_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.shared_experts.up_proj.biases": "model-00105.safetensors",
+    "model.layers.53.post_attention_layernorm.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.kv_a_layernorm.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.kv_a_proj_with_mqa.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.kv_a_proj_with_mqa.scales": "model-00105.safetensors",
+    "model.layers.53.self_attn.kv_a_proj_with_mqa.biases": "model-00105.safetensors",
+    "model.layers.53.self_attn.o_proj.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.o_proj.scales": "model-00105.safetensors",
+    "model.layers.53.self_attn.o_proj.biases": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_a_layernorm.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_a_proj.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_a_proj.scales": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_a_proj.biases": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_b_proj.weight": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_b_proj.scales": "model-00105.safetensors",
+    "model.layers.53.self_attn.q_b_proj.biases": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.gate_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.gate_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.gate_proj.biases": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.down_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.down_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.down_proj.biases": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.up_proj.weight": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.up_proj.scales": "model-00105.safetensors",
+    "model.layers.53.mlp.switch_mlp.up_proj.biases": "model-00105.safetensors",
+    "model.layers.53.self_attn.embed_q.weight": "model-00106.safetensors",
+    "model.layers.53.self_attn.embed_q.scales": "model-00106.safetensors",
+    "model.layers.53.self_attn.embed_q.biases": "model-00106.safetensors",
+    "model.layers.53.self_attn.unembed_out.weight": "model-00106.safetensors",
+    "model.layers.53.self_attn.unembed_out.scales": "model-00106.safetensors",
+    "model.layers.53.self_attn.unembed_out.biases": "model-00106.safetensors",
+    "model.layers.54.input_layernorm.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.gate.e_score_correction_bias": "model-00107.safetensors",
+    "model.layers.54.mlp.gate.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.down_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.down_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.down_proj.biases": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.gate_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.gate_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.gate_proj.biases": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.up_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.up_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.shared_experts.up_proj.biases": "model-00107.safetensors",
+    "model.layers.54.post_attention_layernorm.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.indexer.k_norm.bias": "model-00107.safetensors",
+    "model.layers.54.self_attn.indexer.k_norm.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.indexer.weights_proj.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.indexer.wk.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.indexer.wq_b.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.kv_a_layernorm.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.kv_a_proj_with_mqa.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.kv_a_proj_with_mqa.scales": "model-00107.safetensors",
+    "model.layers.54.self_attn.kv_a_proj_with_mqa.biases": "model-00107.safetensors",
+    "model.layers.54.self_attn.o_proj.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.o_proj.scales": "model-00107.safetensors",
+    "model.layers.54.self_attn.o_proj.biases": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_a_layernorm.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_a_proj.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_a_proj.scales": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_a_proj.biases": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_b_proj.weight": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_b_proj.scales": "model-00107.safetensors",
+    "model.layers.54.self_attn.q_b_proj.biases": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.gate_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.gate_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.gate_proj.biases": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.down_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.down_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.down_proj.biases": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.up_proj.weight": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.up_proj.scales": "model-00107.safetensors",
+    "model.layers.54.mlp.switch_mlp.up_proj.biases": "model-00107.safetensors",
+    "model.layers.54.self_attn.embed_q.weight": "model-00108.safetensors",
+    "model.layers.54.self_attn.embed_q.scales": "model-00108.safetensors",
+    "model.layers.54.self_attn.embed_q.biases": "model-00108.safetensors",
+    "model.layers.54.self_attn.unembed_out.weight": "model-00108.safetensors",
+    "model.layers.54.self_attn.unembed_out.scales": "model-00108.safetensors",
+    "model.layers.54.self_attn.unembed_out.biases": "model-00108.safetensors",
+    "model.layers.55.input_layernorm.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.gate.e_score_correction_bias": "model-00109.safetensors",
+    "model.layers.55.mlp.gate.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.down_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.down_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.down_proj.biases": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.gate_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.gate_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.gate_proj.biases": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.up_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.up_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.shared_experts.up_proj.biases": "model-00109.safetensors",
+    "model.layers.55.post_attention_layernorm.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.kv_a_layernorm.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.kv_a_proj_with_mqa.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.kv_a_proj_with_mqa.scales": "model-00109.safetensors",
+    "model.layers.55.self_attn.kv_a_proj_with_mqa.biases": "model-00109.safetensors",
+    "model.layers.55.self_attn.o_proj.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.o_proj.scales": "model-00109.safetensors",
+    "model.layers.55.self_attn.o_proj.biases": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_a_layernorm.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_a_proj.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_a_proj.scales": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_a_proj.biases": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_b_proj.weight": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_b_proj.scales": "model-00109.safetensors",
+    "model.layers.55.self_attn.q_b_proj.biases": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.gate_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.gate_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.gate_proj.biases": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.down_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.down_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.down_proj.biases": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.up_proj.weight": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.up_proj.scales": "model-00109.safetensors",
+    "model.layers.55.mlp.switch_mlp.up_proj.biases": "model-00109.safetensors",
+    "model.layers.55.self_attn.embed_q.weight": "model-00110.safetensors",
+    "model.layers.55.self_attn.embed_q.scales": "model-00110.safetensors",
+    "model.layers.55.self_attn.embed_q.biases": "model-00110.safetensors",
+    "model.layers.55.self_attn.unembed_out.weight": "model-00110.safetensors",
+    "model.layers.55.self_attn.unembed_out.scales": "model-00110.safetensors",
+    "model.layers.55.self_attn.unembed_out.biases": "model-00110.safetensors",
+    "model.layers.56.input_layernorm.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.gate.e_score_correction_bias": "model-00111.safetensors",
+    "model.layers.56.mlp.gate.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.down_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.down_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.down_proj.biases": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.gate_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.gate_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.gate_proj.biases": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.up_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.up_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.shared_experts.up_proj.biases": "model-00111.safetensors",
+    "model.layers.56.post_attention_layernorm.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.kv_a_layernorm.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.kv_a_proj_with_mqa.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.kv_a_proj_with_mqa.scales": "model-00111.safetensors",
+    "model.layers.56.self_attn.kv_a_proj_with_mqa.biases": "model-00111.safetensors",
+    "model.layers.56.self_attn.o_proj.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.o_proj.scales": "model-00111.safetensors",
+    "model.layers.56.self_attn.o_proj.biases": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_a_layernorm.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_a_proj.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_a_proj.scales": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_a_proj.biases": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_b_proj.weight": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_b_proj.scales": "model-00111.safetensors",
+    "model.layers.56.self_attn.q_b_proj.biases": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.gate_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.gate_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.gate_proj.biases": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.down_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.down_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.down_proj.biases": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.up_proj.weight": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.up_proj.scales": "model-00111.safetensors",
+    "model.layers.56.mlp.switch_mlp.up_proj.biases": "model-00111.safetensors",
+    "model.layers.56.self_attn.embed_q.weight": "model-00112.safetensors",
+    "model.layers.56.self_attn.embed_q.scales": "model-00112.safetensors",
+    "model.layers.56.self_attn.embed_q.biases": "model-00112.safetensors",
+    "model.layers.56.self_attn.unembed_out.weight": "model-00112.safetensors",
+    "model.layers.56.self_attn.unembed_out.scales": "model-00112.safetensors",
+    "model.layers.56.self_attn.unembed_out.biases": "model-00112.safetensors",
+    "model.layers.57.input_layernorm.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.gate.e_score_correction_bias": "model-00113.safetensors",
+    "model.layers.57.mlp.gate.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.down_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.down_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.down_proj.biases": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.gate_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.gate_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.gate_proj.biases": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.up_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.up_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.shared_experts.up_proj.biases": "model-00113.safetensors",
+    "model.layers.57.post_attention_layernorm.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.kv_a_layernorm.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.kv_a_proj_with_mqa.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.kv_a_proj_with_mqa.scales": "model-00113.safetensors",
+    "model.layers.57.self_attn.kv_a_proj_with_mqa.biases": "model-00113.safetensors",
+    "model.layers.57.self_attn.o_proj.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.o_proj.scales": "model-00113.safetensors",
+    "model.layers.57.self_attn.o_proj.biases": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_a_layernorm.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_a_proj.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_a_proj.scales": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_a_proj.biases": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_b_proj.weight": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_b_proj.scales": "model-00113.safetensors",
+    "model.layers.57.self_attn.q_b_proj.biases": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.gate_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.gate_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.gate_proj.biases": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.down_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.down_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.down_proj.biases": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.up_proj.weight": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.up_proj.scales": "model-00113.safetensors",
+    "model.layers.57.mlp.switch_mlp.up_proj.biases": "model-00113.safetensors",
+    "model.layers.57.self_attn.embed_q.weight": "model-00114.safetensors",
+    "model.layers.57.self_attn.embed_q.scales": "model-00114.safetensors",
+    "model.layers.57.self_attn.embed_q.biases": "model-00114.safetensors",
+    "model.layers.57.self_attn.unembed_out.weight": "model-00114.safetensors",
+    "model.layers.57.self_attn.unembed_out.scales": "model-00114.safetensors",
+    "model.layers.57.self_attn.unembed_out.biases": "model-00114.safetensors",
+    "model.layers.58.input_layernorm.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.gate.e_score_correction_bias": "model-00115.safetensors",
+    "model.layers.58.mlp.gate.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.down_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.down_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.down_proj.biases": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.gate_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.gate_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.gate_proj.biases": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.up_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.up_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.shared_experts.up_proj.biases": "model-00115.safetensors",
+    "model.layers.58.post_attention_layernorm.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.indexer.k_norm.bias": "model-00115.safetensors",
+    "model.layers.58.self_attn.indexer.k_norm.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.indexer.weights_proj.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.indexer.wk.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.indexer.wq_b.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.kv_a_layernorm.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.kv_a_proj_with_mqa.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.kv_a_proj_with_mqa.scales": "model-00115.safetensors",
+    "model.layers.58.self_attn.kv_a_proj_with_mqa.biases": "model-00115.safetensors",
+    "model.layers.58.self_attn.o_proj.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.o_proj.scales": "model-00115.safetensors",
+    "model.layers.58.self_attn.o_proj.biases": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_a_layernorm.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_a_proj.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_a_proj.scales": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_a_proj.biases": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_b_proj.weight": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_b_proj.scales": "model-00115.safetensors",
+    "model.layers.58.self_attn.q_b_proj.biases": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.gate_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.gate_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.gate_proj.biases": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.down_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.down_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.down_proj.biases": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.up_proj.weight": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.up_proj.scales": "model-00115.safetensors",
+    "model.layers.58.mlp.switch_mlp.up_proj.biases": "model-00115.safetensors",
+    "model.layers.58.self_attn.embed_q.weight": "model-00116.safetensors",
+    "model.layers.58.self_attn.embed_q.scales": "model-00116.safetensors",
+    "model.layers.58.self_attn.embed_q.biases": "model-00116.safetensors",
+    "model.layers.58.self_attn.unembed_out.weight": "model-00116.safetensors",
+    "model.layers.58.self_attn.unembed_out.scales": "model-00116.safetensors",
+    "model.layers.58.self_attn.unembed_out.biases": "model-00116.safetensors",
+    "model.layers.59.input_layernorm.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.gate.e_score_correction_bias": "model-00117.safetensors",
+    "model.layers.59.mlp.gate.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.down_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.down_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.down_proj.biases": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.gate_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.gate_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.gate_proj.biases": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.up_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.up_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.shared_experts.up_proj.biases": "model-00117.safetensors",
+    "model.layers.59.post_attention_layernorm.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.kv_a_layernorm.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.kv_a_proj_with_mqa.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.kv_a_proj_with_mqa.scales": "model-00117.safetensors",
+    "model.layers.59.self_attn.kv_a_proj_with_mqa.biases": "model-00117.safetensors",
+    "model.layers.59.self_attn.o_proj.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.o_proj.scales": "model-00117.safetensors",
+    "model.layers.59.self_attn.o_proj.biases": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_a_layernorm.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_a_proj.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_a_proj.scales": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_a_proj.biases": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_b_proj.weight": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_b_proj.scales": "model-00117.safetensors",
+    "model.layers.59.self_attn.q_b_proj.biases": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.gate_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.gate_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.gate_proj.biases": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.down_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.down_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.down_proj.biases": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.up_proj.weight": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.up_proj.scales": "model-00117.safetensors",
+    "model.layers.59.mlp.switch_mlp.up_proj.biases": "model-00117.safetensors",
+    "model.layers.59.self_attn.embed_q.weight": "model-00118.safetensors",
+    "model.layers.59.self_attn.embed_q.scales": "model-00118.safetensors",
+    "model.layers.59.self_attn.embed_q.biases": "model-00118.safetensors",
+    "model.layers.59.self_attn.unembed_out.weight": "model-00118.safetensors",
+    "model.layers.59.self_attn.unembed_out.scales": "model-00118.safetensors",
+    "model.layers.59.self_attn.unembed_out.biases": "model-00118.safetensors",
+    "model.layers.60.input_layernorm.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.gate.e_score_correction_bias": "model-00119.safetensors",
+    "model.layers.60.mlp.gate.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.down_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.down_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.down_proj.biases": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.gate_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.gate_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.gate_proj.biases": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.up_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.up_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.shared_experts.up_proj.biases": "model-00119.safetensors",
+    "model.layers.60.post_attention_layernorm.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.kv_a_layernorm.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.kv_a_proj_with_mqa.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.kv_a_proj_with_mqa.scales": "model-00119.safetensors",
+    "model.layers.60.self_attn.kv_a_proj_with_mqa.biases": "model-00119.safetensors",
+    "model.layers.60.self_attn.o_proj.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.o_proj.scales": "model-00119.safetensors",
+    "model.layers.60.self_attn.o_proj.biases": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_a_layernorm.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_a_proj.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_a_proj.scales": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_a_proj.biases": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_b_proj.weight": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_b_proj.scales": "model-00119.safetensors",
+    "model.layers.60.self_attn.q_b_proj.biases": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.gate_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.gate_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.gate_proj.biases": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.down_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.down_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.down_proj.biases": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.up_proj.weight": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.up_proj.scales": "model-00119.safetensors",
+    "model.layers.60.mlp.switch_mlp.up_proj.biases": "model-00119.safetensors",
+    "model.layers.60.self_attn.embed_q.weight": "model-00120.safetensors",
+    "model.layers.60.self_attn.embed_q.scales": "model-00120.safetensors",
+    "model.layers.60.self_attn.embed_q.biases": "model-00120.safetensors",
+    "model.layers.60.self_attn.unembed_out.weight": "model-00120.safetensors",
+    "model.layers.60.self_attn.unembed_out.scales": "model-00120.safetensors",
+    "model.layers.60.self_attn.unembed_out.biases": "model-00120.safetensors",
+    "model.layers.61.input_layernorm.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.gate.e_score_correction_bias": "model-00121.safetensors",
+    "model.layers.61.mlp.gate.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.down_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.down_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.down_proj.biases": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.gate_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.gate_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.gate_proj.biases": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.up_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.up_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.shared_experts.up_proj.biases": "model-00121.safetensors",
+    "model.layers.61.post_attention_layernorm.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.kv_a_layernorm.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.kv_a_proj_with_mqa.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.kv_a_proj_with_mqa.scales": "model-00121.safetensors",
+    "model.layers.61.self_attn.kv_a_proj_with_mqa.biases": "model-00121.safetensors",
+    "model.layers.61.self_attn.o_proj.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.o_proj.scales": "model-00121.safetensors",
+    "model.layers.61.self_attn.o_proj.biases": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_a_layernorm.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_a_proj.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_a_proj.scales": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_a_proj.biases": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_b_proj.weight": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_b_proj.scales": "model-00121.safetensors",
+    "model.layers.61.self_attn.q_b_proj.biases": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.gate_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.gate_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.gate_proj.biases": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.down_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.down_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.down_proj.biases": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.up_proj.weight": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.up_proj.scales": "model-00121.safetensors",
+    "model.layers.61.mlp.switch_mlp.up_proj.biases": "model-00121.safetensors",
+    "model.layers.61.self_attn.embed_q.weight": "model-00122.safetensors",
+    "model.layers.61.self_attn.embed_q.scales": "model-00122.safetensors",
+    "model.layers.61.self_attn.embed_q.biases": "model-00122.safetensors",
+    "model.layers.61.self_attn.unembed_out.weight": "model-00122.safetensors",
+    "model.layers.61.self_attn.unembed_out.scales": "model-00122.safetensors",
+    "model.layers.61.self_attn.unembed_out.biases": "model-00122.safetensors",
+    "model.layers.62.input_layernorm.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.gate.e_score_correction_bias": "model-00123.safetensors",
+    "model.layers.62.mlp.gate.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.down_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.down_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.down_proj.biases": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.gate_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.gate_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.gate_proj.biases": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.up_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.up_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.shared_experts.up_proj.biases": "model-00123.safetensors",
+    "model.layers.62.post_attention_layernorm.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.indexer.k_norm.bias": "model-00123.safetensors",
+    "model.layers.62.self_attn.indexer.k_norm.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.indexer.weights_proj.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.indexer.wk.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.indexer.wq_b.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.kv_a_layernorm.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.kv_a_proj_with_mqa.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.kv_a_proj_with_mqa.scales": "model-00123.safetensors",
+    "model.layers.62.self_attn.kv_a_proj_with_mqa.biases": "model-00123.safetensors",
+    "model.layers.62.self_attn.o_proj.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.o_proj.scales": "model-00123.safetensors",
+    "model.layers.62.self_attn.o_proj.biases": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_a_layernorm.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_a_proj.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_a_proj.scales": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_a_proj.biases": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_b_proj.weight": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_b_proj.scales": "model-00123.safetensors",
+    "model.layers.62.self_attn.q_b_proj.biases": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.gate_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.gate_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.gate_proj.biases": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.down_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.down_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.down_proj.biases": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.up_proj.weight": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.up_proj.scales": "model-00123.safetensors",
+    "model.layers.62.mlp.switch_mlp.up_proj.biases": "model-00123.safetensors",
+    "model.layers.62.self_attn.embed_q.weight": "model-00124.safetensors",
+    "model.layers.62.self_attn.embed_q.scales": "model-00124.safetensors",
+    "model.layers.62.self_attn.embed_q.biases": "model-00124.safetensors",
+    "model.layers.62.self_attn.unembed_out.weight": "model-00124.safetensors",
+    "model.layers.62.self_attn.unembed_out.scales": "model-00124.safetensors",
+    "model.layers.62.self_attn.unembed_out.biases": "model-00124.safetensors",
+    "model.layers.63.input_layernorm.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.gate.e_score_correction_bias": "model-00125.safetensors",
+    "model.layers.63.mlp.gate.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.down_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.down_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.down_proj.biases": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.gate_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.gate_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.gate_proj.biases": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.up_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.up_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.shared_experts.up_proj.biases": "model-00125.safetensors",
+    "model.layers.63.post_attention_layernorm.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.kv_a_layernorm.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.kv_a_proj_with_mqa.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.kv_a_proj_with_mqa.scales": "model-00125.safetensors",
+    "model.layers.63.self_attn.kv_a_proj_with_mqa.biases": "model-00125.safetensors",
+    "model.layers.63.self_attn.o_proj.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.o_proj.scales": "model-00125.safetensors",
+    "model.layers.63.self_attn.o_proj.biases": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_a_layernorm.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_a_proj.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_a_proj.scales": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_a_proj.biases": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_b_proj.weight": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_b_proj.scales": "model-00125.safetensors",
+    "model.layers.63.self_attn.q_b_proj.biases": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.gate_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.gate_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.gate_proj.biases": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.down_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.down_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.down_proj.biases": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.up_proj.weight": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.up_proj.scales": "model-00125.safetensors",
+    "model.layers.63.mlp.switch_mlp.up_proj.biases": "model-00125.safetensors",
+    "model.layers.63.self_attn.embed_q.weight": "model-00126.safetensors",
+    "model.layers.63.self_attn.embed_q.scales": "model-00126.safetensors",
+    "model.layers.63.self_attn.embed_q.biases": "model-00126.safetensors",
+    "model.layers.63.self_attn.unembed_out.weight": "model-00126.safetensors",
+    "model.layers.63.self_attn.unembed_out.scales": "model-00126.safetensors",
+    "model.layers.63.self_attn.unembed_out.biases": "model-00126.safetensors",
+    "model.layers.64.input_layernorm.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.gate.e_score_correction_bias": "model-00127.safetensors",
+    "model.layers.64.mlp.gate.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.down_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.down_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.down_proj.biases": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.gate_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.gate_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.gate_proj.biases": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.up_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.up_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.shared_experts.up_proj.biases": "model-00127.safetensors",
+    "model.layers.64.post_attention_layernorm.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.kv_a_layernorm.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.kv_a_proj_with_mqa.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.kv_a_proj_with_mqa.scales": "model-00127.safetensors",
+    "model.layers.64.self_attn.kv_a_proj_with_mqa.biases": "model-00127.safetensors",
+    "model.layers.64.self_attn.o_proj.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.o_proj.scales": "model-00127.safetensors",
+    "model.layers.64.self_attn.o_proj.biases": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_a_layernorm.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_a_proj.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_a_proj.scales": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_a_proj.biases": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_b_proj.weight": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_b_proj.scales": "model-00127.safetensors",
+    "model.layers.64.self_attn.q_b_proj.biases": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.gate_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.gate_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.gate_proj.biases": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.down_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.down_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.down_proj.biases": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.up_proj.weight": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.up_proj.scales": "model-00127.safetensors",
+    "model.layers.64.mlp.switch_mlp.up_proj.biases": "model-00127.safetensors",
+    "model.layers.64.self_attn.embed_q.weight": "model-00128.safetensors",
+    "model.layers.64.self_attn.embed_q.scales": "model-00128.safetensors",
+    "model.layers.64.self_attn.embed_q.biases": "model-00128.safetensors",
+    "model.layers.64.self_attn.unembed_out.weight": "model-00128.safetensors",
+    "model.layers.64.self_attn.unembed_out.scales": "model-00128.safetensors",
+    "model.layers.64.self_attn.unembed_out.biases": "model-00128.safetensors",
+    "model.layers.65.input_layernorm.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.gate.e_score_correction_bias": "model-00129.safetensors",
+    "model.layers.65.mlp.gate.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.down_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.down_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.down_proj.biases": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.gate_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.gate_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.gate_proj.biases": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.up_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.up_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.shared_experts.up_proj.biases": "model-00129.safetensors",
+    "model.layers.65.post_attention_layernorm.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.kv_a_layernorm.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.kv_a_proj_with_mqa.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.kv_a_proj_with_mqa.scales": "model-00129.safetensors",
+    "model.layers.65.self_attn.kv_a_proj_with_mqa.biases": "model-00129.safetensors",
+    "model.layers.65.self_attn.o_proj.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.o_proj.scales": "model-00129.safetensors",
+    "model.layers.65.self_attn.o_proj.biases": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_a_layernorm.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_a_proj.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_a_proj.scales": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_a_proj.biases": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_b_proj.weight": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_b_proj.scales": "model-00129.safetensors",
+    "model.layers.65.self_attn.q_b_proj.biases": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.gate_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.gate_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.gate_proj.biases": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.down_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.down_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.down_proj.biases": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.up_proj.weight": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.up_proj.scales": "model-00129.safetensors",
+    "model.layers.65.mlp.switch_mlp.up_proj.biases": "model-00129.safetensors",
+    "model.layers.65.self_attn.embed_q.weight": "model-00130.safetensors",
+    "model.layers.65.self_attn.embed_q.scales": "model-00130.safetensors",
+    "model.layers.65.self_attn.embed_q.biases": "model-00130.safetensors",
+    "model.layers.65.self_attn.unembed_out.weight": "model-00130.safetensors",
+    "model.layers.65.self_attn.unembed_out.scales": "model-00130.safetensors",
+    "model.layers.65.self_attn.unembed_out.biases": "model-00130.safetensors",
+    "model.layers.66.input_layernorm.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.gate.e_score_correction_bias": "model-00131.safetensors",
+    "model.layers.66.mlp.gate.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.down_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.down_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.down_proj.biases": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.gate_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.gate_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.gate_proj.biases": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.up_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.up_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.shared_experts.up_proj.biases": "model-00131.safetensors",
+    "model.layers.66.post_attention_layernorm.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.indexer.k_norm.bias": "model-00131.safetensors",
+    "model.layers.66.self_attn.indexer.k_norm.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.indexer.weights_proj.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.indexer.wk.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.indexer.wq_b.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.kv_a_layernorm.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.kv_a_proj_with_mqa.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.kv_a_proj_with_mqa.scales": "model-00131.safetensors",
+    "model.layers.66.self_attn.kv_a_proj_with_mqa.biases": "model-00131.safetensors",
+    "model.layers.66.self_attn.o_proj.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.o_proj.scales": "model-00131.safetensors",
+    "model.layers.66.self_attn.o_proj.biases": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_a_layernorm.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_a_proj.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_a_proj.scales": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_a_proj.biases": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_b_proj.weight": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_b_proj.scales": "model-00131.safetensors",
+    "model.layers.66.self_attn.q_b_proj.biases": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.gate_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.gate_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.gate_proj.biases": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.down_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.down_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.down_proj.biases": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.up_proj.weight": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.up_proj.scales": "model-00131.safetensors",
+    "model.layers.66.mlp.switch_mlp.up_proj.biases": "model-00131.safetensors",
+    "model.layers.66.self_attn.embed_q.weight": "model-00132.safetensors",
+    "model.layers.66.self_attn.embed_q.scales": "model-00132.safetensors",
+    "model.layers.66.self_attn.embed_q.biases": "model-00132.safetensors",
+    "model.layers.66.self_attn.unembed_out.weight": "model-00132.safetensors",
+    "model.layers.66.self_attn.unembed_out.scales": "model-00132.safetensors",
+    "model.layers.66.self_attn.unembed_out.biases": "model-00132.safetensors",
+    "model.layers.67.input_layernorm.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.gate.e_score_correction_bias": "model-00133.safetensors",
+    "model.layers.67.mlp.gate.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.down_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.down_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.down_proj.biases": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.gate_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.gate_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.gate_proj.biases": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.up_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.up_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.shared_experts.up_proj.biases": "model-00133.safetensors",
+    "model.layers.67.post_attention_layernorm.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.kv_a_layernorm.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.kv_a_proj_with_mqa.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.kv_a_proj_with_mqa.scales": "model-00133.safetensors",
+    "model.layers.67.self_attn.kv_a_proj_with_mqa.biases": "model-00133.safetensors",
+    "model.layers.67.self_attn.o_proj.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.o_proj.scales": "model-00133.safetensors",
+    "model.layers.67.self_attn.o_proj.biases": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_a_layernorm.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_a_proj.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_a_proj.scales": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_a_proj.biases": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_b_proj.weight": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_b_proj.scales": "model-00133.safetensors",
+    "model.layers.67.self_attn.q_b_proj.biases": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.gate_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.gate_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.gate_proj.biases": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.down_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.down_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.down_proj.biases": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.up_proj.weight": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.up_proj.scales": "model-00133.safetensors",
+    "model.layers.67.mlp.switch_mlp.up_proj.biases": "model-00133.safetensors",
+    "model.layers.67.self_attn.embed_q.weight": "model-00134.safetensors",
+    "model.layers.67.self_attn.embed_q.scales": "model-00134.safetensors",
+    "model.layers.67.self_attn.embed_q.biases": "model-00134.safetensors",
+    "model.layers.67.self_attn.unembed_out.weight": "model-00134.safetensors",
+    "model.layers.67.self_attn.unembed_out.scales": "model-00134.safetensors",
+    "model.layers.67.self_attn.unembed_out.biases": "model-00134.safetensors",
+    "model.layers.68.input_layernorm.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.gate.e_score_correction_bias": "model-00135.safetensors",
+    "model.layers.68.mlp.gate.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.down_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.down_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.down_proj.biases": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.gate_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.gate_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.gate_proj.biases": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.up_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.up_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.shared_experts.up_proj.biases": "model-00135.safetensors",
+    "model.layers.68.post_attention_layernorm.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.kv_a_layernorm.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.kv_a_proj_with_mqa.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.kv_a_proj_with_mqa.scales": "model-00135.safetensors",
+    "model.layers.68.self_attn.kv_a_proj_with_mqa.biases": "model-00135.safetensors",
+    "model.layers.68.self_attn.o_proj.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.o_proj.scales": "model-00135.safetensors",
+    "model.layers.68.self_attn.o_proj.biases": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_a_layernorm.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_a_proj.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_a_proj.scales": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_a_proj.biases": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_b_proj.weight": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_b_proj.scales": "model-00135.safetensors",
+    "model.layers.68.self_attn.q_b_proj.biases": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.gate_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.gate_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.gate_proj.biases": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.down_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.down_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.down_proj.biases": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.up_proj.weight": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.up_proj.scales": "model-00135.safetensors",
+    "model.layers.68.mlp.switch_mlp.up_proj.biases": "model-00135.safetensors",
+    "model.layers.68.self_attn.embed_q.weight": "model-00136.safetensors",
+    "model.layers.68.self_attn.embed_q.scales": "model-00136.safetensors",
+    "model.layers.68.self_attn.embed_q.biases": "model-00136.safetensors",
+    "model.layers.68.self_attn.unembed_out.weight": "model-00136.safetensors",
+    "model.layers.68.self_attn.unembed_out.scales": "model-00136.safetensors",
+    "model.layers.68.self_attn.unembed_out.biases": "model-00136.safetensors",
+    "model.layers.69.input_layernorm.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.gate.e_score_correction_bias": "model-00137.safetensors",
+    "model.layers.69.mlp.gate.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.down_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.down_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.down_proj.biases": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.gate_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.gate_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.gate_proj.biases": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.up_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.up_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.shared_experts.up_proj.biases": "model-00137.safetensors",
+    "model.layers.69.post_attention_layernorm.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.kv_a_layernorm.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.kv_a_proj_with_mqa.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.kv_a_proj_with_mqa.scales": "model-00137.safetensors",
+    "model.layers.69.self_attn.kv_a_proj_with_mqa.biases": "model-00137.safetensors",
+    "model.layers.69.self_attn.o_proj.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.o_proj.scales": "model-00137.safetensors",
+    "model.layers.69.self_attn.o_proj.biases": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_a_layernorm.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_a_proj.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_a_proj.scales": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_a_proj.biases": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_b_proj.weight": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_b_proj.scales": "model-00137.safetensors",
+    "model.layers.69.self_attn.q_b_proj.biases": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.gate_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.gate_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.gate_proj.biases": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.down_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.down_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.down_proj.biases": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.up_proj.weight": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.up_proj.scales": "model-00137.safetensors",
+    "model.layers.69.mlp.switch_mlp.up_proj.biases": "model-00137.safetensors",
+    "model.layers.69.self_attn.embed_q.weight": "model-00138.safetensors",
+    "model.layers.69.self_attn.embed_q.scales": "model-00138.safetensors",
+    "model.layers.69.self_attn.embed_q.biases": "model-00138.safetensors",
+    "model.layers.69.self_attn.unembed_out.weight": "model-00138.safetensors",
+    "model.layers.69.self_attn.unembed_out.scales": "model-00138.safetensors",
+    "model.layers.69.self_attn.unembed_out.biases": "model-00138.safetensors",
+    "model.layers.70.input_layernorm.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.gate.e_score_correction_bias": "model-00139.safetensors",
+    "model.layers.70.mlp.gate.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.down_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.down_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.down_proj.biases": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.gate_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.gate_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.gate_proj.biases": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.up_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.up_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.shared_experts.up_proj.biases": "model-00139.safetensors",
+    "model.layers.70.post_attention_layernorm.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.indexer.k_norm.bias": "model-00139.safetensors",
+    "model.layers.70.self_attn.indexer.k_norm.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.indexer.weights_proj.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.indexer.wk.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.indexer.wq_b.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.kv_a_layernorm.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.kv_a_proj_with_mqa.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.kv_a_proj_with_mqa.scales": "model-00139.safetensors",
+    "model.layers.70.self_attn.kv_a_proj_with_mqa.biases": "model-00139.safetensors",
+    "model.layers.70.self_attn.o_proj.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.o_proj.scales": "model-00139.safetensors",
+    "model.layers.70.self_attn.o_proj.biases": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_a_layernorm.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_a_proj.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_a_proj.scales": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_a_proj.biases": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_b_proj.weight": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_b_proj.scales": "model-00139.safetensors",
+    "model.layers.70.self_attn.q_b_proj.biases": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.gate_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.gate_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.gate_proj.biases": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.down_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.down_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.down_proj.biases": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.up_proj.weight": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.up_proj.scales": "model-00139.safetensors",
+    "model.layers.70.mlp.switch_mlp.up_proj.biases": "model-00139.safetensors",
+    "model.layers.70.self_attn.embed_q.weight": "model-00140.safetensors",
+    "model.layers.70.self_attn.embed_q.scales": "model-00140.safetensors",
+    "model.layers.70.self_attn.embed_q.biases": "model-00140.safetensors",
+    "model.layers.70.self_attn.unembed_out.weight": "model-00140.safetensors",
+    "model.layers.70.self_attn.unembed_out.scales": "model-00140.safetensors",
+    "model.layers.70.self_attn.unembed_out.biases": "model-00140.safetensors",
+    "model.layers.71.input_layernorm.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.gate.e_score_correction_bias": "model-00141.safetensors",
+    "model.layers.71.mlp.gate.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.down_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.down_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.down_proj.biases": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.gate_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.gate_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.gate_proj.biases": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.up_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.up_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.shared_experts.up_proj.biases": "model-00141.safetensors",
+    "model.layers.71.post_attention_layernorm.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.kv_a_layernorm.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.kv_a_proj_with_mqa.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.kv_a_proj_with_mqa.scales": "model-00141.safetensors",
+    "model.layers.71.self_attn.kv_a_proj_with_mqa.biases": "model-00141.safetensors",
+    "model.layers.71.self_attn.o_proj.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.o_proj.scales": "model-00141.safetensors",
+    "model.layers.71.self_attn.o_proj.biases": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_a_layernorm.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_a_proj.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_a_proj.scales": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_a_proj.biases": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_b_proj.weight": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_b_proj.scales": "model-00141.safetensors",
+    "model.layers.71.self_attn.q_b_proj.biases": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.gate_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.gate_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.gate_proj.biases": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.down_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.down_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.down_proj.biases": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.up_proj.weight": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.up_proj.scales": "model-00141.safetensors",
+    "model.layers.71.mlp.switch_mlp.up_proj.biases": "model-00141.safetensors",
+    "model.layers.71.self_attn.embed_q.weight": "model-00142.safetensors",
+    "model.layers.71.self_attn.embed_q.scales": "model-00142.safetensors",
+    "model.layers.71.self_attn.embed_q.biases": "model-00142.safetensors",
+    "model.layers.71.self_attn.unembed_out.weight": "model-00142.safetensors",
+    "model.layers.71.self_attn.unembed_out.scales": "model-00142.safetensors",
+    "model.layers.71.self_attn.unembed_out.biases": "model-00142.safetensors",
+    "model.layers.72.input_layernorm.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.gate.e_score_correction_bias": "model-00143.safetensors",
+    "model.layers.72.mlp.gate.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.down_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.down_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.down_proj.biases": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.gate_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.gate_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.gate_proj.biases": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.up_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.up_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.shared_experts.up_proj.biases": "model-00143.safetensors",
+    "model.layers.72.post_attention_layernorm.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.kv_a_layernorm.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.kv_a_proj_with_mqa.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.kv_a_proj_with_mqa.scales": "model-00143.safetensors",
+    "model.layers.72.self_attn.kv_a_proj_with_mqa.biases": "model-00143.safetensors",
+    "model.layers.72.self_attn.o_proj.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.o_proj.scales": "model-00143.safetensors",
+    "model.layers.72.self_attn.o_proj.biases": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_a_layernorm.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_a_proj.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_a_proj.scales": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_a_proj.biases": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_b_proj.weight": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_b_proj.scales": "model-00143.safetensors",
+    "model.layers.72.self_attn.q_b_proj.biases": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.gate_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.gate_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.gate_proj.biases": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.down_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.down_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.down_proj.biases": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.up_proj.weight": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.up_proj.scales": "model-00143.safetensors",
+    "model.layers.72.mlp.switch_mlp.up_proj.biases": "model-00143.safetensors",
+    "model.layers.72.self_attn.embed_q.weight": "model-00144.safetensors",
+    "model.layers.72.self_attn.embed_q.scales": "model-00144.safetensors",
+    "model.layers.72.self_attn.embed_q.biases": "model-00144.safetensors",
+    "model.layers.72.self_attn.unembed_out.weight": "model-00144.safetensors",
+    "model.layers.72.self_attn.unembed_out.scales": "model-00144.safetensors",
+    "model.layers.72.self_attn.unembed_out.biases": "model-00144.safetensors",
+    "model.layers.73.input_layernorm.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.gate.e_score_correction_bias": "model-00145.safetensors",
+    "model.layers.73.mlp.gate.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.down_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.down_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.down_proj.biases": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.gate_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.gate_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.gate_proj.biases": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.up_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.up_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.shared_experts.up_proj.biases": "model-00145.safetensors",
+    "model.layers.73.post_attention_layernorm.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.kv_a_layernorm.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.kv_a_proj_with_mqa.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.kv_a_proj_with_mqa.scales": "model-00145.safetensors",
+    "model.layers.73.self_attn.kv_a_proj_with_mqa.biases": "model-00145.safetensors",
+    "model.layers.73.self_attn.o_proj.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.o_proj.scales": "model-00145.safetensors",
+    "model.layers.73.self_attn.o_proj.biases": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_a_layernorm.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_a_proj.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_a_proj.scales": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_a_proj.biases": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_b_proj.weight": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_b_proj.scales": "model-00145.safetensors",
+    "model.layers.73.self_attn.q_b_proj.biases": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.gate_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.gate_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.gate_proj.biases": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.down_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.down_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.down_proj.biases": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.up_proj.weight": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.up_proj.scales": "model-00145.safetensors",
+    "model.layers.73.mlp.switch_mlp.up_proj.biases": "model-00145.safetensors",
+    "model.layers.73.self_attn.embed_q.weight": "model-00146.safetensors",
+    "model.layers.73.self_attn.embed_q.scales": "model-00146.safetensors",
+    "model.layers.73.self_attn.embed_q.biases": "model-00146.safetensors",
+    "model.layers.73.self_attn.unembed_out.weight": "model-00146.safetensors",
+    "model.layers.73.self_attn.unembed_out.scales": "model-00146.safetensors",
+    "model.layers.73.self_attn.unembed_out.biases": "model-00146.safetensors",
+    "model.layers.74.input_layernorm.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.gate.e_score_correction_bias": "model-00147.safetensors",
+    "model.layers.74.mlp.gate.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.down_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.down_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.down_proj.biases": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.gate_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.gate_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.gate_proj.biases": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.up_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.up_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.shared_experts.up_proj.biases": "model-00147.safetensors",
+    "model.layers.74.post_attention_layernorm.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.indexer.k_norm.bias": "model-00147.safetensors",
+    "model.layers.74.self_attn.indexer.k_norm.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.indexer.weights_proj.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.indexer.wk.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.indexer.wq_b.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.kv_a_layernorm.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.kv_a_proj_with_mqa.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.kv_a_proj_with_mqa.scales": "model-00147.safetensors",
+    "model.layers.74.self_attn.kv_a_proj_with_mqa.biases": "model-00147.safetensors",
+    "model.layers.74.self_attn.o_proj.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.o_proj.scales": "model-00147.safetensors",
+    "model.layers.74.self_attn.o_proj.biases": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_a_layernorm.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_a_proj.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_a_proj.scales": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_a_proj.biases": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_b_proj.weight": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_b_proj.scales": "model-00147.safetensors",
+    "model.layers.74.self_attn.q_b_proj.biases": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.gate_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.gate_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.gate_proj.biases": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.down_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.down_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.down_proj.biases": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.up_proj.weight": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.up_proj.scales": "model-00147.safetensors",
+    "model.layers.74.mlp.switch_mlp.up_proj.biases": "model-00147.safetensors",
+    "model.layers.74.self_attn.embed_q.weight": "model-00148.safetensors",
+    "model.layers.74.self_attn.embed_q.scales": "model-00148.safetensors",
+    "model.layers.74.self_attn.embed_q.biases": "model-00148.safetensors",
+    "model.layers.74.self_attn.unembed_out.weight": "model-00148.safetensors",
+    "model.layers.74.self_attn.unembed_out.scales": "model-00148.safetensors",
+    "model.layers.74.self_attn.unembed_out.biases": "model-00148.safetensors",
+    "model.layers.75.input_layernorm.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.gate.e_score_correction_bias": "model-00149.safetensors",
+    "model.layers.75.mlp.gate.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.down_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.down_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.down_proj.biases": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.gate_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.gate_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.gate_proj.biases": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.up_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.up_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.shared_experts.up_proj.biases": "model-00149.safetensors",
+    "model.layers.75.post_attention_layernorm.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.kv_a_layernorm.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.kv_a_proj_with_mqa.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.kv_a_proj_with_mqa.scales": "model-00149.safetensors",
+    "model.layers.75.self_attn.kv_a_proj_with_mqa.biases": "model-00149.safetensors",
+    "model.layers.75.self_attn.o_proj.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.o_proj.scales": "model-00149.safetensors",
+    "model.layers.75.self_attn.o_proj.biases": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_a_layernorm.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_a_proj.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_a_proj.scales": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_a_proj.biases": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_b_proj.weight": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_b_proj.scales": "model-00149.safetensors",
+    "model.layers.75.self_attn.q_b_proj.biases": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.gate_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.gate_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.gate_proj.biases": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.down_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.down_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.down_proj.biases": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.up_proj.weight": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.up_proj.scales": "model-00149.safetensors",
+    "model.layers.75.mlp.switch_mlp.up_proj.biases": "model-00149.safetensors",
+    "model.layers.75.self_attn.embed_q.weight": "model-00150.safetensors",
+    "model.layers.75.self_attn.embed_q.scales": "model-00150.safetensors",
+    "model.layers.75.self_attn.embed_q.biases": "model-00150.safetensors",
+    "model.layers.75.self_attn.unembed_out.weight": "model-00150.safetensors",
+    "model.layers.75.self_attn.unembed_out.scales": "model-00150.safetensors",
+    "model.layers.75.self_attn.unembed_out.biases": "model-00150.safetensors",
+    "model.layers.76.input_layernorm.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.gate.e_score_correction_bias": "model-00151.safetensors",
+    "model.layers.76.mlp.gate.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.down_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.down_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.down_proj.biases": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.gate_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.gate_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.gate_proj.biases": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.up_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.up_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.shared_experts.up_proj.biases": "model-00151.safetensors",
+    "model.layers.76.post_attention_layernorm.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.kv_a_layernorm.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.kv_a_proj_with_mqa.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.kv_a_proj_with_mqa.scales": "model-00151.safetensors",
+    "model.layers.76.self_attn.kv_a_proj_with_mqa.biases": "model-00151.safetensors",
+    "model.layers.76.self_attn.o_proj.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.o_proj.scales": "model-00151.safetensors",
+    "model.layers.76.self_attn.o_proj.biases": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_a_layernorm.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_a_proj.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_a_proj.scales": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_a_proj.biases": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_b_proj.weight": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_b_proj.scales": "model-00151.safetensors",
+    "model.layers.76.self_attn.q_b_proj.biases": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.gate_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.gate_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.gate_proj.biases": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.down_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.down_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.down_proj.biases": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.up_proj.weight": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.up_proj.scales": "model-00151.safetensors",
+    "model.layers.76.mlp.switch_mlp.up_proj.biases": "model-00151.safetensors",
+    "model.layers.76.self_attn.embed_q.weight": "model-00152.safetensors",
+    "model.layers.76.self_attn.embed_q.scales": "model-00152.safetensors",
+    "model.layers.76.self_attn.embed_q.biases": "model-00152.safetensors",
+    "model.layers.76.self_attn.unembed_out.weight": "model-00152.safetensors",
+    "model.layers.76.self_attn.unembed_out.scales": "model-00152.safetensors",
+    "model.layers.76.self_attn.unembed_out.biases": "model-00152.safetensors",
+    "model.layers.77.input_layernorm.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.gate.e_score_correction_bias": "model-00153.safetensors",
+    "model.layers.77.mlp.gate.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.down_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.down_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.down_proj.biases": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.gate_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.gate_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.gate_proj.biases": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.up_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.up_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.shared_experts.up_proj.biases": "model-00153.safetensors",
+    "model.layers.77.post_attention_layernorm.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.kv_a_layernorm.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.kv_a_proj_with_mqa.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.kv_a_proj_with_mqa.scales": "model-00153.safetensors",
+    "model.layers.77.self_attn.kv_a_proj_with_mqa.biases": "model-00153.safetensors",
+    "model.layers.77.self_attn.o_proj.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.o_proj.scales": "model-00153.safetensors",
+    "model.layers.77.self_attn.o_proj.biases": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_a_layernorm.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_a_proj.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_a_proj.scales": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_a_proj.biases": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_b_proj.weight": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_b_proj.scales": "model-00153.safetensors",
+    "model.layers.77.self_attn.q_b_proj.biases": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.gate_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.gate_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.gate_proj.biases": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.down_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.down_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.down_proj.biases": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.up_proj.weight": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.up_proj.scales": "model-00153.safetensors",
+    "model.layers.77.mlp.switch_mlp.up_proj.biases": "model-00153.safetensors",
+    "model.layers.77.self_attn.embed_q.weight": "model-00154.safetensors",
+    "model.layers.77.self_attn.embed_q.scales": "model-00154.safetensors",
+    "model.layers.77.self_attn.embed_q.biases": "model-00154.safetensors",
+    "model.layers.77.self_attn.unembed_out.weight": "model-00154.safetensors",
+    "model.layers.77.self_attn.unembed_out.scales": "model-00154.safetensors",
+    "model.layers.77.self_attn.unembed_out.biases": "model-00154.safetensors"
+  }
+}
\ No newline at end of file
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..aba40197a4cdb5607f4ab7a05fb0a4ee8054fd6d
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
+size 20217442
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..e375fa0a4e16660ce0e2026e39374d35dad4c079
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,33 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": false,
+  "do_lower_case": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "[MASK]",
+    "[gMASK]",
+    "[sMASK]",
+    "<sop>",
+    "<eop>",
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|observation|>",
+    "<|begin_of_image|>",
+    "<|end_of_image|>",
+    "<|begin_of_video|>",
+    "<|end_of_video|>",
+    "<|begin_of_audio|>",
+    "<|end_of_audio|>",
+    "<|begin_of_transcription|>",
+    "<|end_of_transcription|>"
+  ],
+  "is_local": true,
+  "model_max_length": 1048576,
+  "model_specific_special_tokens": {},
+  "pad_token": "<|endoftext|>",
+  "padding_side": "left",
+  "remove_space": false,
+  "tokenizer_class": "TokenizersBackend"
+}