final

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

BOARD.md

@@ -0,0 +1,37 @@
+# Mach-1-Ternary-Additive-35B — benchmark board
+
+Payload: integer L1-ball trellis expert codes + per-wavefront gamma scales +
+continuous fp16 su/sv side-streams; 64-level integer-lattice spine; int5-g64
+head; int4 embed — every weight matmul is add/subtract-only. Ship gate:
+decode.py-primitive reconstruction == served checkpoint (bf16 rounding,
+layers 0/20/39).
+
+Protocol: same-harness EvalScope 1.9.1 + vLLM, PrismML App. B (thinking mode,
+temp 1.0, top_p 0.95, top_k 20, PrismML token tiers, AIME mean-of-8, IFEval
+prompt-strict, IFBench prompt-loose). Retention = 100 x score / BF16 teacher
+(Qwen3.6-35B-A3B), same harness. tau2-bench = fixed external user-simulator
+(Qwen3.6-35B BF16, greedy), single pass, identical for every model.
+
+## Flagship payload (12/12)
+
+| benchmark | score | teacher | retention |
+|---|---|---|---|
+| AIME25 @8 | 87.50 | 88.33 | **99.1%** |
+| AIME26 @8 | 89.58 | 90.00 | **99.5%** |
+| MATH-500 | 98.00 | 98.60 | **99.4%** |
+| GSM8K | 94.69 | 96.21 | 98.4% |
+| MBPP+ | 94.44 | 96.03 | 98.3% |
+| HumanEval+ | 92.68 | 95.12 | 97.4% |
+| MMLU-Redux | 89.18 | 92.68 | 96.2% |
+| IFEval | 83.75 (n=5) | 89.05 | 94.0% |
+| MuSR | 61.77 | 66.66 | 92.7% |
+| BFCL-v3 | 68.97 | 74.98 | 92.0% |
+| tau2-bench | 71.58 | 79.51 | 90.0% |
+| IFBench | 54.08 | 64.97 | 83.2% |
+| **mean retention** | | | **95.0%** |
+
+## Read-quality notes
+- Multi-read cells quote the mean over all reads with n; single reads are n=1.
+- IFEval strict single-read spread measured ~2-3 pts; tau2 complete-read
+  spread up to ~6 on some artifacts — sub-point deltas are ties.
+- Expert payload 6.207 GB.

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Alibaba Cloud
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

MANIFEST.json

@@ -0,0 +1,25 @@
+{
+ "payload": "rdsl64j_s300",
+ "base": "Qwen/Qwen3.6-35B-A3B",
+ "categories": {
+  "experts_packed": 42,
+  "ne_packed": 41,
+  "head_packed": 8,
+  "embed_packed": 2,
+  "extras": 361,
+  "mtp_dropped": 17
+ },
+ "packed_GB": {
+  "experts": 6.2083,
+  "ne_spine": 0.7039,
+  "head": 0.3337,
+  "embed_int4": 0.2875,
+  "embed_lut4_alternative_container": 0.5086
+ },
+ "embed_note": "embed ships in two interchangeable containers (int4 codes; lossless lut4); deployed payload counts one (int4)",
+ "extras_GB": 0.0523,
+ "n_e2e_norm_overrides": 131,
+ "text_payload_GB": 7.5335,
+ "text_bpw": 1.6978,
+ "codec_reference": "packed/experts/codec.json"
+}

README.md

@@ -0,0 +1,58 @@
+<!-- Theme-aware embed. If your renderer ignores <picture>, use the plain
+     image line underneath instead and delete this block. -->
+
+![Screenshot 2026-08-03 at 10.32.36 AM](https://cdn-uploads.huggingface.co/production/uploads/680868b984cac4b136ea7900/0YxMKj31WMJT1NwbVAGjj.png)
+
+<!-- Plain fallback:
+![Capability retention vs BF16 teacher](assets/benchmarks-light.png)
+-->
+
+## Benchmarks
+
+
+
+| | Mean retention, 12 benchmarks |
+| --- | ---: |
+| **Mach-1 Small** | **95.0%** |
+| Ternary Bonsai 27B (PrismML) | 93.6% |
+| Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
+
+### Per-benchmark Retention
+
+| Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
+| --- | ---: | ---: | ---: |
+| AIME26 | **99.5%** | 92.7% | 67.7% |
+| MATH-500 | **99.4%** | 98.2% | 95.6% |
+| AIME25 | **99.1%** | 91.7% | 67.2% |
+| GSM8K | 98.4% | **100.2%** | 97.3% |
+| MBPP+ | 98.3% | **98.4%** | 92.2% |
+| HumanEval+ | 97.4% | **98.7%** | 94.1% |
+| MMLU-Redux | 96.2% | 94.0% | **96.9%** |
+| IFEval | 94.0% | 89.8% | **95.5%** |
+| MuSR | **92.7%** | 91.6% | 91.1% |
+| BFCL-v3 | 92.0% | **98.9%** | 95.7% |
+| τ²-bench | 90.0% | **91.2%** | 73.1% |
+| IFBench | **83.2%** | 77.7% | 61.3% |
+| **Mean** | **95.0%** | **93.6%** | **85.6%** |
+
+Mach-1 Small's own scores and teacher scores:
+
+| Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
+| --- | ---: | ---: | ---: |
+| AIME26 | 89.58 | 90.00 | 99.5% |
+| MATH-500 | 98.00 | 98.60 | 99.4% |
+| AIME25 | 87.50 | 88.33 | 99.1% |
+| GSM8K | 94.69 | 96.21 | 98.4% |
+| MBPP+ | 94.44 | 96.03 | 98.3% |
+| HumanEval+ | 92.68 | 95.12 | 97.4% |
+| MMLU-Redux | 89.18 | 92.68 | 96.2% |
+| IFEval | 83.75 | 89.05 | 94.0% |
+| MuSR | 61.77 | 66.66 | 92.7% |
+| BFCL-v3 | 68.97 | 74.98 | 92.0% |
+| τ²-bench | 71.58 | 79.51 | 90.0% |
+| IFBench | 54.08 | 64.97 | 83.2% |
+
+### Speed
+
+![Screenshot 2026-08-03 at 11.55.47 AM](https://cdn-uploads.huggingface.co/production/uploads/680868b984cac4b136ea7900/lYFfUCU1QWrGjM23vbb66.png)
+

RELEASE_NOTES.md

@@ -0,0 +1,15 @@
+# Release notes
+
+Current payload:
+
+- Experts: integer L1-ball trellis codes (K=1.5, V=8; book:
+  `packed/experts/codebook.safetensors`) + `wave_gamma` per-wavefront scales +
+  continuous fp16 `su`/`sv` side-streams. Expert files carry NO Wscale — `sv`
+  is the complete scale (decode with `wscale=None`); `wave_gamma` applies in
+  rotated space before the `su`/`sv` Hadamards.
+- Spine: 64-level integer lattice in the K=4 trellis. Head: int5-g64.
+  Embedding: int4 + exact-overwrite exception list.
+- Ship gate: decode.py-primitive reconstruction == the served checkpoint
+  (bf16 rounding, layers 0/20/39 x 4 experts x 3 projs).
+- Full measured board and retention table: see BOARD.md. Expert payload
+  6.207 GB.

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,119 @@
+{
+  "architectures": [
+    "Qwen3_5MoeForConditionalGeneration"
+  ],
+  "image_token_id": 248056,
+  "model_type": "qwen3_5_moe",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": 248044,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_5_moe_text",
+    "moe_intermediate_size": 512,
+    "mtp_num_hidden_layers": 0,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_experts": 256,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 2,
+    "output_router_logits": false,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "router_aux_loss_coef": 0.001,
+    "shared_expert_intermediate_size": 512,
+    "tie_word_embeddings": false,
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.1",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 27,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_5_moe",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053
+}

decode.py

@@ -0,0 +1,649 @@
+#!/usr/bin/env python3
+"""Standalone numpy decoder for the packed Mach-1 checkpoint.
+
+Packed layout (HF repo):
+  packed/experts/L{LL}.safetensors -- per expert e, proj in gate/up/down:
+      e{e}.{proj}.trellis  int16  [ntiles, 16*K]      (trellis bitstream, ~K bpw)
+      e{e}.{proj}.su       fp16   [n]
+      e{e}.{proj}.sv       fp16   [m]   (Wscale absorbed)
+    or (demoted K1 experts)
+      e{e}.{proj}.SU       int8   [n];  e{e}.{proj}.SV  int8 [m];  e{e}.{proj}.Wscale fp16 []
+    plus, when the manifest carries "basis" (shared cold-expert low-rank basis):
+      basis.{proj}.A       fp16   [r, n];  basis.{proj}.B  fp16  [m, r]
+      e{e}.{proj}.c        fp16   [r]   (demoted experts only)
+    metadata["manifest"]: {"cb2":..., "cb1":..., "demoted":[e,...], "geom":{proj:[m,n]},
+                           optional "basis": {"r":r, "dtype":"fp16", "shared_A_gu":false}}
+  packed/experts/codebook.safetensors: tlut fp16 [512, 2] (shared by K2/K1; the tlut
+    depends only on (tlut_bits=9, V=2) -- K only changes trellis transitions).
+  packed/ne/L{LL}.safetensors (zero-padded LNN) -- codec
+    "canon_rht_bitshift_trellis_intlattice"; no manifest key. Per-file metadata:
+    cb_params {K:4,L:16,V:2,tlut_bits:9,quantlut_sym,td 16x16} + dims
+    {name: [m0, n0, m, n]}. Keys per tensor:
+      {name}|trellis int16, {name}|SU int8 [n], {name}|SV int8 [m], {name}|Wscale fp16
+    tier-shared codebook: packed/ne/tlut.safetensors  tlut fp16 [512, 2]
+  packed/ne/{L{i}|head_c{c}of8}.safetensors (single-digit L{i}: older builds) --
+    transform-free tier:
+      {name}|packed uint8 [B, T*K/8], {name}|gscale fp16 [B, T/128], {name}|lut fp16 [4096]
+    metadata["manifest"]: {"codec":"lloyd_bitshift_trellis","L":12,"group":128,
+                           "pattern":[K]*4,"tensors":{name:{shape:[m0,n0],transposed:bool}}}
+  packed/head/head_c{c}of8.safetensors -- codec "int5g64_packed":
+      LMHEADCHUNK:{r0}:{r1}|qp uint8 [rows, n/8*5], |gscale fp16 [rows, n/64]
+      (+ optional |prot_rows int32 / |prot_dense exact rows); metadata dims {name:[rows,n]}
+  packed/ne/embed_int4.safetensors (decode_embed(bits=4)) --
+    affine int4-g64: q_packed uint8 [rows, hid/2], mn/mx fp16 [rows, hid/64]
+  packed/ne/embed_packed.safetensors (8-bpw lossless Lloyd-LUT container;
+    decode_embed_packed) and embed_int3.safetensors (older int3):
+      q_packed uint8 [rows, hid*3/8], mn fp16 [rows, hid/64], mx fp16 [rows, hid/64]
+
+Usage:
+    from decode import (decode_trellis, decode_expert_layer, decode_ne_shard,
+                        decode_head, decode_embed_packed, decode_embed)
+"""
+import json
+import math
+import os
+
+import numpy as np
+
+HIDDEN, INTER, NEXP, NLAYERS = 2048, 512, 256, 40
+NHEADC = 8
+CB2 = dict(K=2, L=16, V=2, tlut_bits=9, decode_mode="quantlut_sym", td_x=16, td_y=16)
+CB1 = dict(K=1, L=16, V=2, tlut_bits=9, decode_mode="quantlut_sym", td_x=16, td_y=16)
+CB4 = dict(K=4, L=16, V=2, tlut_bits=9, decode_mode="quantlut_sym", td_x=16, td_y=16)
+
+def padto(d):
+    """Dim -> padded dim: identity for powers of two, else next power of two."""
+    if d > 0 and (d & (d - 1)) == 0:
+        return d
+    return 1 << math.ceil(math.log2(d))
+
+
+# ============================================================================ #
+# Expert tier. Format:
+#   * A weight matrix is padded to (m, n) = (padto(m0), padto(n0)) and cut into
+#     td_x x td_y tiles, row-major over the (m/td_x, n/td_y) grid.
+#   * Each tile is a length-T scalar sequence (T = td_x*td_y, row-major inside the
+#     tile) produced by an L-bit shift register: the register emits one V-vector
+#     per step (T/V steps), then shifts in K*V fresh bits. State recurrence:
+#         reg_i = ((reg_{i-1} << K*V) | fresh_i) & (2^L - 1)
+#   * Bitstream per tile: the L bits of reg_0 (MSB first), then the K*V fresh bits
+#     of each later step (MSB first) -- T*K bits total, packed into BIG-ENDIAN
+#     16-bit words. The last L-K*V register bits are not stored: the sequence is
+#     tail-biting, they wrap around to the start of the stream.
+#   * A register state s maps to a V-vector through a hashed symmetric LUT
+#     ("quantlut_sym"): with p = s*(s+1) exact-integer,
+#         row(s)   = (p >> (16 - tlut_bits - 1)) & (2^tlut_bits - 1)
+#         vec(s)   = tlut[row(s)] with component 0 negated iff bit 15 of p is set
+#   * Un-rotation (two-sided RHT): with H_d the orthonormal Walsh-Hadamard matrix
+#     (Sylvester order, scaled 1/sqrt(d); symmetric),
+#         W = diag(sv) . H_m . Wunit . H_n . diag(su),  restricted to [:m0, :n0]
+#     computed as: FWHT each row over n then scale columns by su, then FWHT each
+#     column over m and scale rows by sv. For K1 packs Wunit is first multiplied
+#     by the scalar `wscale`.
+# The operation order above (and in _np_hadamard) is part of the format: fp32
+# elementwise add/sub/mul/div are IEEE-exact, so following it reproduces the
+# encoder's decode exactly.
+# ============================================================================ #
+_FULL_LUT_CACHE = {}
+
+
+def _np_full_lut(tlut, L, tlut_bits):
+    """Expand the persisted [2^tlut_bits, V] codebook to the full [2^L, V] fp32
+    decoder table per the hashed-symmetric-LUT rule above."""
+    small = np.asarray(tlut, np.float32)
+    s = np.arange(1 << L, dtype=np.int64)
+    p = s * (s + 1)                                        # exact in int64
+    row = (p >> (16 - tlut_bits - 1)) & ((1 << tlut_bits) - 1)
+    table = small[row].copy()                              # [2^L, V]
+    table[:, 0] *= (1 - ((p >> 15) & 1) * 2).astype(np.float32)
+    return table
+
+
+def _np_full_lut_cached(tlut, L, tlut_bits):
+    key = (L, tlut_bits, np.asarray(tlut).tobytes())
+    if key not in _FULL_LUT_CACHE:
+        _FULL_LUT_CACHE[key] = _np_full_lut(tlut, L, tlut_bits)
+    return _FULL_LUT_CACHE[key]
+
+
+def _np_rate_bits(T, K, V):
+    """(bits per shift step, bits per tile). K need not be an integer: the only
+    rate constraint is that K*V and K*T are whole numbers of bits."""
+    step, nbits = float(K) * V, float(K) * T
+    if step != int(step) or nbits != int(nbits):
+        raise ValueError(f"rate K={K} with V={V}, T={T} needs whole-bit steps "
+                         f"(K*V={step}) and a whole-bit tile (K*T={nbits})")
+    return int(step), int(nbits)
+
+
+def _np_unpack_trellis(stream, T, L, K, V):
+    """Packed bitstream [rows, T*K/16] (u)int16 -> register states [rows, T//V] int32."""
+    words = np.ascontiguousarray(stream)
+    if words.dtype != np.uint16:
+        words = words.view(np.uint16)
+    step, nbits = _np_rate_bits(T, K, V)
+    if step > L:
+        raise ValueError(f"K*V={step} exceeds register width L={L}")
+    rows, nstep = words.shape[0], T // V
+    bits = ((words[:, :, None].astype(np.int64) >> np.arange(15, -1, -1)) & 1)
+    bits = bits.reshape(rows, -1)[:, :nbits]               # MSB-first big-endian words
+    bits = np.concatenate([bits, bits[:, :L - step]], axis=1)   # tail-biting wrap
+    seed_w = 1 << np.arange(L - 1, -1, -1, dtype=np.int64)
+    step_w = 1 << np.arange(step - 1, -1, -1, dtype=np.int64)
+    fresh = bits[:, L:L + (nstep - 1) * step].reshape(rows, nstep - 1, step) @ step_w
+    states = np.empty((rows, nstep), np.int32)
+    reg = bits[:, :L] @ seed_w
+    states[:, 0] = reg
+    mask = (1 << L) - 1
+    for i in range(1, nstep):
+        reg = ((reg << step) & mask) | fresh[:, i - 1]
+        states[:, i] = reg
+    return states
+
+
+def _np_recons(states, table, m, n, td_x, td_y):
+    """Register states [ntiles, T//V] + full LUT -> codebook-unit weights [m, n] fp32
+    (V scalars per state, row-major tiles, row-major tile grid)."""
+    vals = table[states]                                   # [ntiles, T//V, V]
+    return np.ascontiguousarray(
+        vals.reshape(m // td_x, n // td_y, td_x, td_y).transpose(0, 2, 1, 3)
+    ).reshape(m, n)
+
+
+def _np_hadamard(x):
+    """Orthonormal Walsh-Hadamard transform (Sylvester order) along the LAST axis:
+    y = FWHT(x) / sqrt(dim). Butterflies pair at stride 1, then 2, 4, ... in fp32,
+    with a single fp32 division by sqrt(dim) after the final pass."""
+    dim = x.shape[-1]
+    if dim & (dim - 1):
+        raise ValueError(f"pure-numpy RHT needs a power-of-2 dim, got {dim}")
+    cur = np.ascontiguousarray(x, dtype=np.float32).reshape(-1, dim)
+    span = 1
+    while span < dim:
+        blk = cur.reshape(-1, dim // (2 * span), 2, span)
+        nxt = np.empty_like(blk)
+        nxt[:, :, 0, :] = blk[:, :, 0, :] + blk[:, :, 1, :]
+        nxt[:, :, 1, :] = blk[:, :, 0, :] - blk[:, :, 1, :]
+        cur = nxt.reshape(-1, dim)
+        span *= 2
+    return (cur / np.float32(np.sqrt(np.float32(dim)))).reshape(x.shape)
+
+
+def decode_trellis(trellis, su, sv, tlut, m0, n0, cb_params, wscale=None, cb=None,
+                   device="cuda"):
+    """Trellis decode -> fp32 [m0, n0].
+
+    su [n] / sv [m] are the RHT vectors over the PADDED dims: int8 +/-1 signs for raw
+    PTQ tensors (then `wscale` must be given), or continuous fp16 vectors for the
+    K2 experts (Wscale absorbed into sv; pass wscale=None).
+    Reverses: unpack states -> tiles -> [m,n] -> *wscale -> *su-side FWHT -> *sv-side
+    FWHT -> unpad. `cb` may carry a prebuilt table (numpy [2^L, V] LUT);
+    anything else is ignored and rebuilt from tlut.
+    """
+    mode = cb_params.get("decode_mode", "quantlut_sym")
+    if mode != "quantlut_sym":
+        raise NotImplementedError(f"numpy decode implements quantlut_sym, got {mode}")
+    td_x, td_y = cb_params["td_x"], cb_params["td_y"]
+    L, K, V = cb_params["L"], cb_params["K"], cb_params["V"]
+    m, n = padto(m0), padto(n0)
+    table = cb if isinstance(cb, np.ndarray) else \
+        _np_full_lut_cached(tlut, L, cb_params["tlut_bits"])
+    states = _np_unpack_trellis(np.asarray(trellis), td_x * td_y, L, K, V)
+    unit = _np_recons(states, table, m, n, td_x, td_y)
+    # the unit tensor is defined at fp16 precision
+    unit = unit.astype(np.float16).astype(np.float32)
+    if wscale is not None:
+        unit = unit * np.float32(wscale)
+    rowside = _np_hadamard(unit) * np.asarray(su, np.float32)      # over n, then *su
+    colside = _np_hadamard(rowside.T) * np.asarray(sv, np.float32)  # over m, then *sv
+    return np.ascontiguousarray(colside.T[:m0, :n0])
+
+
+# ============================================================================ #
+# packed-dir glue: whole-layer / whole-shard reconstruction.
+# ============================================================================ #
+def _read_safetensors_np(path):
+    """Read a pack file. v3 files carry a zstd sidecar (`__zsc__` + metadata["zsc"])
+    holding every non-code-stream tensor byte-exactly; expand it transparently."""
+    from safetensors import safe_open
+    out = {}
+    with safe_open(path, framework="numpy") as fh:
+        meta = fh.metadata() or {}
+        for k in fh.keys():
+            out[k] = fh.get_tensor(k)
+    if "__zsc__" in out:
+        import zstandard
+        man = json.loads(meta["zsc"])
+        buf = zstandard.ZstdDecompressor().decompress(
+            out.pop("__zsc__").tobytes(), max_output_size=man["raw_len"])
+        for key, dt, shape, off, nb in man["entries"]:
+            out[key] = np.frombuffer(buf, dtype=np.dtype(dt),
+                                     count=nb // np.dtype(dt).itemsize,
+                                     offset=off).reshape(shape)
+    return out, meta
+
+
+_ST_TORCH_DTYPES = {"BF16": "bfloat16", "F16": "float16", "F32": "float32",
+                    "F64": "float64", "I8": "int8", "U8": "uint8", "I16": "int16",
+                    "I32": "int32", "I64": "int64", "BOOL": "bool"}
+
+
+def read_safetensors_torch(path):
+    """Torch-side v2/v3 reader for dtype-opaque files (bf16). v3 files hold ONLY
+    `__zsc__`; entries carry safetensors dtype tokens ("BF16", ...).
+    Returns ({name: torch.Tensor}, metadata)."""
+    import torch
+    from safetensors import safe_open
+    out = {}
+    with safe_open(path, framework="pt") as fh:
+        meta = fh.metadata() or {}
+        for k in fh.keys():
+            out[k] = fh.get_tensor(k)
+    if "__zsc__" in out:
+        import zstandard
+        man = json.loads(meta["zsc"])
+        buf = zstandard.ZstdDecompressor().decompress(
+            out.pop("__zsc__").numpy().tobytes(), max_output_size=man["raw_len"])
+        for key, dt, shape, off, nb in man["entries"]:
+            if dt in _ST_TORCH_DTYPES:                       # safetensors dtype token
+                t = torch.frombuffer(bytearray(buf[off:off + nb]),
+                                     dtype=getattr(torch, _ST_TORCH_DTYPES[dt]))
+                out[key] = t.reshape(shape)
+            else:                                            # numpy dtype str
+                a = np.frombuffer(buf, dtype=np.dtype(dt),
+                                  count=nb // np.dtype(dt).itemsize, offset=off)
+                out[key] = torch.from_numpy(a.reshape(shape).copy())
+    return out, meta
+
+
+def decode_expert_layer(packed_dir, layer, device="cuda"):
+    """Reassemble fused gate_up_proj [256,1024,2048] fp32 + down_proj [256,2048,512].
+    K2 experts decode with continuous su/sv (no wscale); demoted K1 experts decode
+    with int8 signs + Wscale, plus the shared low-rank basis residual (B*c)@A when
+    the manifest carries "basis"."""
+    path = os.path.join(packed_dir, "experts", f"L{layer:02d}.safetensors")
+    t, meta = _read_safetensors_np(path)
+    # chunked shards (metadata fields includes wave_gamma) route to
+    # decode_expert_layer_v3t
+    if "wave_gamma" in (meta.get("fields") or ""):
+        return decode_expert_layer_v3t(packed_dir, layer)
+    man = json.loads(meta["manifest"])
+    tlut, _ = _read_safetensors_np(os.path.join(packed_dir, "experts", "codebook.safetensors"))
+    tlut = tlut["tlut"]
+    # K only changes transitions, so the [2^L, V] LUT is shared
+    cb2 = cb1 = _np_full_lut_cached(tlut, man.get("cb2", CB2)["L"],
+                                        man.get("cb2", CB2)["tlut_bits"])
+    demoted = set(man["demoted"])
+    geom = {p: tuple(v) for p, v in man["geom"].items()}   # proj -> (m, n)
+    basis = man.get("basis")
+    if basis is not None:              # shared cold-expert low-rank basis
+        bA = {p: t[f"basis.{p}.A"].astype(np.float32) for p in geom}
+        bB = {p: t[f"basis.{p}.B"].astype(np.float32) for p in geom}
+    gate_up = np.empty((NEXP, 2 * INTER, HIDDEN), np.float32)
+    down = np.empty((NEXP, HIDDEN, INTER), np.float32)
+    for e in range(NEXP):
+        for proj, dst in (("gate", gate_up[e, :INTER]), ("up", gate_up[e, INTER:]),
+                          ("down", down[e])):
+            m0, n0 = geom[proj]
+            if e in demoted:
+                ws = float(np.asarray(t[f"e{e}.{proj}.Wscale"]).ravel()[0])
+                w = decode_trellis(t[f"e{e}.{proj}.trellis"], t[f"e{e}.{proj}.SU"],
+                                   t[f"e{e}.{proj}.SV"], tlut, m0, n0, man.get("cb1", CB1),
+                                   wscale=ws, cb=cb1, device=device)
+                if basis is not None:
+                    # rs[m,n] = sum_r B[m,r]*c[r]*A[r,n], fp32, added BEFORE any
+                    # bf16 cast -- the op order is part of the format
+                    c = t[f"e{e}.{proj}.c"].astype(np.float32)
+                    w = w + (bB[proj] * c[None, :]) @ bA[proj]
+            else:
+                w = decode_trellis(t[f"e{e}.{proj}.trellis"], t[f"e{e}.{proj}.su"],
+                                   t[f"e{e}.{proj}.sv"], tlut, m0, n0, man.get("cb2", CB2),
+                                   wscale=None, cb=cb2, device=device)
+            dst[:] = w
+    return {"gate_up_proj": gate_up, "down_proj": down}
+
+
+# ============================================================================ #
+# NE transform-free tier: Lloyd bitshift trellis L=12, uniform K, group-128 fp16
+# scales, per-matrix 4096-entry fp16 LUT.
+# ============================================================================ #
+NELL_L = 12
+
+
+def _nell_bits_to_states(packed, T, k, L=NELL_L):
+    """Inverse of the encoder's states_to_bits: step t's k NEW trellis bits are stored
+    MSB-first, bytes are big-endian packbits. State recurrence (s_{-1} = 0):
+    s_t = ((s_{t-1} << k) | b_t) & (2^L - 1)."""
+    B = packed.shape[0]
+    bits = np.unpackbits(packed, axis=1, count=T * k).astype(np.int64)
+    mask = (1 << L) - 1
+    states = np.empty((B, T), np.int32)
+    s = np.zeros(B, np.int64)
+    pos = 0
+    for t in range(T):
+        b = np.zeros(B, np.int64)
+        for j in range(k):
+            b = (b << 1) | bits[:, pos]
+            pos += 1
+        s = ((s << k) | b) & mask
+        states[:, t] = s
+    return states
+
+
+def decode_ne_ll_tensor(t, man, name):
+    """One transform-free NE matrix -> fp32 [m0, n0]: fp16 LUT gathered as fp32,
+    * per-group fp16 scale in fp32, transpose back."""
+    g, k = man["group"], man["pattern"][0]
+    geom = man["tensors"][name]
+    m0, n0 = geom["shape"]
+    B, T = (n0, m0) if geom["transposed"] else (m0, n0)
+    lut = t[f"{name}|lut"].astype(np.float32)
+    gs = t[f"{name}|gscale"].astype(np.float32)
+    states = _nell_bits_to_states(t[f"{name}|packed"], T, k)
+    W = lut[states] * np.repeat(gs, g, axis=1)
+    return np.ascontiguousarray(W.T) if geom["transposed"] else W
+
+
+def decode_ne_shard_canon(t, meta, packed_dir, device="cuda", subdir="ne"):
+    """NE spine shard (codec canon_rht_bitshift_trellis_intlattice, zero-padded
+    L00-L39 files). No manifest key: per-file metadata holds cb_params and dims
+    (name -> [m0, n0, m, n]); keys are <tensor>|{trellis,SU,SV,Wscale} with int8
+    sign SU/SV + scalar Wscale; the tier-shared codebook is <subdir>/tlut.safetensors."""
+    cbp = json.loads(meta["cb_params"])
+    dims = json.loads(meta["dims"])
+    tlut, _ = _read_safetensors_np(os.path.join(packed_dir, subdir, "tlut.safetensors"))
+    tlut = tlut["tlut"]
+    cb = _np_full_lut_cached(tlut, cbp["L"], cbp["tlut_bits"])
+    out = {}
+    for name in sorted({key.rsplit("|", 1)[0] for key in t}):
+        m0, n0 = dims[name][0], dims[name][1]
+        out[name] = decode_trellis(t[f"{name}|trellis"], t[f"{name}|SU"], t[f"{name}|SV"],
+                                   tlut, m0, n0, cbp,
+                                   wscale=float(np.asarray(t[f"{name}|Wscale"]).ravel()[0]),
+                                   cb=cb, device=device)
+    return out
+
+
+def decode_ne_shard(packed_dir, shard, device="cuda", subdir="ne"):
+    """Decode every NE tensor in one shard -> {name: fp32 [m0,n0]}. Dispatches on the
+    shard's own metadata. subdir picks the size variant: "ne" (default) or "ne-4bit"."""
+    path = os.path.join(packed_dir, subdir, f"{shard}.safetensors")
+    t, meta = _read_safetensors_np(path)
+    if "manifest" not in meta:
+        codec = meta.get("codec")
+        assert codec == "canon_rht_bitshift_trellis_intlattice", \
+            f"NE shard {shard}: no manifest and unknown codec {codec!r}"
+        return decode_ne_shard_canon(t, meta, packed_dir, device=device, subdir=subdir)
+    man = json.loads(meta["manifest"])
+    if man.get("codec") == "lloyd_bitshift_trellis":
+        return {name: decode_ne_ll_tensor(t, man, name) for name in man["tensors"]}
+    cb_params = man["cb"]
+    tlut, _ = _read_safetensors_np(os.path.join(packed_dir, "ne", "codebook.safetensors"))
+    tlut = tlut["tlut"]
+    cb = _np_full_lut_cached(tlut, cb_params["L"], cb_params["tlut_bits"])
+    out = {}
+    for name, geom in man["tensors"].items():
+        out[name] = decode_trellis(t[f"{name}|trellis"], t[f"{name}|SU"], t[f"{name}|SV"],
+                                   tlut, geom["m0"], geom["n0"], cb_params,
+                                   wscale=float(np.asarray(t[f"{name}|Wscale"]).ravel()[0]),
+                                   cb=cb, device=device)
+    return out
+
+
+# ============================================================================ #
+# Embedding: int{3,4} asymmetric group codes, and the lossless Lloyd-LUT container.
+# ============================================================================ #
+def pack_embed_q(q, bits=3):
+    """q uint8 [rows, hid] with values < 2**bits -> packed uint8 [rows, hid*bits/8]."""
+    rows, hid = q.shape
+    b = np.unpackbits(q[..., None], axis=-1, count=8)[..., 8 - bits:]   # [rows,hid,bits] MSB-first
+    return np.packbits(b.reshape(rows, hid * bits), axis=1)
+
+
+def unpack_embed_q(packed, hid, bits=3):
+    rows = packed.shape[0]
+    b = np.unpackbits(packed, axis=1, count=hid * bits).reshape(rows, hid, bits)
+    q = np.zeros((rows, hid), np.uint8)
+    for j in range(bits):
+        q = (q << 1) | b[..., j]
+    return q
+
+
+def decode_embed(packed_dir, bits=3, group=None):
+    """embed_int{bits} codes -> fp32 [rows, hid]. mn/mx stored fp16, step computed
+    in fp32 exactly as at encode time. `group` is inferred from the stored shapes
+    when not given, so g64 and g128 packs decode identically.
+    Optional exception tensors: elements listed in exc_idx int32 (flat index) are
+    overwritten with the exact bf16 bit patterns in exc_bits uint16 (expanded to
+    fp32 here)."""
+    path = os.path.join(packed_dir, "ne", f"embed_int{bits}.safetensors")
+    t, meta = _read_safetensors_np(path)
+    mn = t["mn"].astype(np.float32)[..., None]              # [rows, hid/g, 1]
+    mx = t["mx"].astype(np.float32)[..., None]
+    rows, ng = mn.shape[0], mn.shape[1]
+    if group is None:
+        group = (t["q_packed"].shape[1] * 8 // bits) // ng
+    hid = ng * group
+    q = unpack_embed_q(t["q_packed"], hid, bits=bits).astype(np.float32)
+    lv = float(2 ** bits - 1)
+    step = np.maximum(mx - mn, 1e-8) / lv
+    dec = (mn + q.reshape(rows, ng, group) * step).reshape(rows, hid)
+    if "exc_idx" in t:
+        vals = (np.asarray(t["exc_bits"]).astype(np.uint32) << 16).view(np.float32)
+        dec.reshape(-1)[np.asarray(t["exc_idx"], dtype=np.int64)] = vals
+    return dec
+
+
+def decode_embed_packed(packed_dir, subdir="ne"):
+    """embed_packed.safetensors (per-group Lloyd LUT + 4-bit nibble codes, chunk keys
+    EMBEDCHUNK:{r0}:{r1}.{codes|lut}) -> fp32 [rows, hid]. Lossless: the lut stores
+    the source bf16 bit patterns. Low nibble = even column. Uses the torch-side
+    reader because the lut is bf16."""
+    path = os.path.join(packed_dir, subdir, "embed_packed.safetensors")
+    t, meta = read_safetensors_torch(path)
+    group = int(meta.get("group", 128))
+    chunks = sorted({k.rsplit(".", 1)[0] for k in t},
+                    key=lambda c: int(c.split(":")[1]))
+    r_end, parts = 0, []
+    for c in chunks:
+        r0, r1 = int(c.split(":")[1]), int(c.split(":")[2])
+        assert r0 == r_end, f"non-contiguous embed chunks at {c}"
+        r_end = r1
+        codes = np.asarray(t[f"{c}.codes"])                  # uint8 [rows, cols/2]
+        lut = t[f"{c}.lut"].float().numpy()                  # [G, 16] exact bf16 values
+        rows, half = codes.shape
+        cols = half * 2
+        q = np.empty((rows, cols), np.uint8)
+        q[:, 0::2] = codes & 0x0F
+        q[:, 1::2] = codes >> 4
+        w = np.take_along_axis(lut, q.reshape(-1, group).astype(np.int64), axis=1)
+        parts.append(w.reshape(rows, cols))
+    return np.concatenate(parts, axis=0)
+
+
+if __name__ == "__main__":
+    import argparse
+    ap = argparse.ArgumentParser(description="spot-verify a packed dir against decoded refs")
+    ap.add_argument("--packed-dir", required=True)
+    ap.add_argument("--ref-experts", default=None)
+    ap.add_argument("--ref-ne", default=None)
+    ap.add_argument("--layers", default="0")
+    ap.add_argument("--ne-shards", default="L0")
+    ap.add_argument("--device", default="cuda")
+    a = ap.parse_args()
+
+    def _re(x, y):
+        x, y = np.asarray(x, np.float32).ravel(), np.asarray(y, np.float32).ravel()
+        return float(np.linalg.norm(x - y) / max(np.linalg.norm(y), 1e-30))
+
+    if a.ref_experts:
+        for L in [int(x) for x in a.layers.split(",") if x.strip()]:
+            dec = decode_expert_layer(a.packed_dir, L, device=a.device)
+            ref, _ = _read_safetensors_np(os.path.join(a.ref_experts, f"L{L:02d}.safetensors"))
+            for key in ("gate_up_proj", "down_proj"):
+                print(f"[experts L{L} {key}] relerr_vs_ref={_re(dec[key], ref[key]):.2e}")
+    if a.ref_ne:
+        for shard in [x for x in a.ne_shards.split(",") if x.strip()]:
+            dec = decode_ne_shard(a.packed_dir, shard, device=a.device)
+            refp = os.path.join(a.ref_ne, f"{shard}.safetensors")
+            ref, _ = _read_safetensors_np(refp) if os.path.exists(refp) else ({}, {})
+            for name, w in dec.items():
+                if name in ref:
+                    print(f"[NE {shard} {name}] relerr_vs_ref={_re(w, ref[name]):.4f}")
+                else:
+                    print(f"[NE {shard} {name}] shape={w.shape} (no ref)")
+
+
+# ============================================================================ #
+# int5-g64 head tier: symmetric int5 codes, one fp16 scale per 64 reduction-dim
+# weights, RAW domain (no rotation). Storage: 8 codes packed into 5 little-endian
+# bytes (code i occupies bits [5i, 5i+5) of the 40-bit block; stored value =
+# q + 16, q in [-16, 15]).
+# Optional protected rows: |prot_rows int32 + |prot_dense bf16 overwrite the
+# listed rows with exact dense values.
+# ============================================================================ #
+def pack_int5(q):
+    """int8 [m, n] in [-16, 15] -> uint8 [m, n//8*5] little-endian 5-bit pack."""
+    m, n = q.shape
+    assert n % 8 == 0, n
+    u = (q.astype(np.int64) + 16).astype(np.uint64)
+    assert u.max() < 32 and u.min() >= 0, (int(u.min()), int(u.max()))
+    blocks = u.reshape(m, n // 8, 8)
+    word = np.zeros((m, n // 8), dtype=np.uint64)
+    for i in range(8):
+        word |= blocks[:, :, i] << np.uint64(5 * i)
+    by = word.astype("<u8").view(np.uint8).reshape(m, n // 8, 8)[:, :, :5]
+    return np.ascontiguousarray(by.reshape(m, n // 8 * 5))
+
+
+def unpack_int5(qp, n):
+    """Inverse of pack_int5 -> int8 [m, n] in [-16, 15]."""
+    m = qp.shape[0]
+    assert qp.shape[1] == n // 8 * 5, (qp.shape, n)
+    by = qp.reshape(m, n // 8, 5)
+    full = np.zeros((m, n // 8, 8), dtype=np.uint8)
+    full[:, :, :5] = by
+    word = full.reshape(m, n // 8 * 8).view("<u8").reshape(m, n // 8)
+    out = np.zeros((m, n // 8, 8), dtype=np.int8)
+    for i in range(8):
+        out[:, :, i] = ((word >> np.uint64(5 * i)) & np.uint64(31)).astype(np.int8) - 16
+    return out.reshape(m, n)
+
+
+def decode_int5g64(qp, gscale, m0, n0, group=64, prot_rows=None, prot_dense=None):
+    """int5-g64 head decode -> fp32 [m0, n0]: W[r, j] = q[r, j] * gscale[r, j // group];
+    protected rows are then overwritten with their exact dense values."""
+    q = unpack_int5(np.asarray(qp), n0).astype(np.float32)
+    s = np.asarray(gscale, dtype=np.float32)
+    W = q * np.repeat(s, group, axis=1)[:, :n0]
+    if prot_rows is not None and len(prot_rows):
+        W[np.asarray(prot_rows, dtype=np.int64)] = np.asarray(prot_dense,
+                                                              dtype=np.float32)
+    return np.ascontiguousarray(W[:m0, :n0])
+
+
+def decode_head(packed_dir, subdir="head"):
+    """int5-g64 lm_head (packed/{subdir}/head_c{c}of8.safetensors, codec
+    "int5g64_packed") -> fp32 [vocab, hid]. Chunk keys LMHEADCHUNK:{r0}:{r1}|{qp|gscale}
+    (+ optional |prot_rows / |prot_dense); each file's dims metadata gives the chunk's
+    [rows, n]; row chunks assemble in r0 order."""
+    d = os.path.join(packed_dir, subdir)
+    files = sorted(f for f in os.listdir(d)
+                   if f.startswith("head_c") and f.endswith(".safetensors"))
+    assert files, f"no head chunk files under {d}"
+    pieces = []
+    for f in files:
+        t, meta = _read_safetensors_np(os.path.join(d, f))
+        group = int(meta.get("group", 64))
+        dims = json.loads(meta["dims"])
+        for name, (m0, n0) in dims.items():
+            r0 = int(name.split(":")[1])
+            w = decode_int5g64(t[f"{name}|qp"], t[f"{name}|gscale"], m0, n0,
+                               group=group, prot_rows=t.get(f"{name}|prot_rows"),
+                               prot_dense=t.get(f"{name}|prot_dense"))
+            pieces.append((r0, w))
+    pieces.sort(key=lambda x: x[0])
+    return np.concatenate([w for _, w in pieces], axis=0)
+
+
+# ============================================================================ #
+# Chunked-container expert decode with per-(expert, wavefront) gammas.
+# Layout: 32-expert CHUNK-STACKED keys  e{c0}.{proj}.{trellis|su|sv|wave_gamma}
+# (c0 in 0,32,...,224), NO Wscale / int8 signs. su/sv are continuous fp16 over
+# the PADDED dims with Wscale absorbed into sv; wave_gamma is fp16 [n_chunk,
+# Mb+Nb] indexed by wave_index_map. Discriminator: file metadata carries
+# fields="trellis|su|sv|wave_gamma".
+# Op order (part of the format):
+#   states -> recons -> cast fp16 -> apply_wave_gamma   (BEFORE both Hadamards)
+#   -> hadamard over n -> * su    (between the Hadamards)
+#   -> transpose -> hadamard over m -> * sv -> transpose -> crop
+# ============================================================================ #
+def wave_index_map(Mb, Nb):
+    """Tile grid [Mb, Nb] -> the LAST wavefront index that wrote each tile
+    (the encoder's starts recurrence, including the schedule's duplicated
+    top-right-starting wave)."""
+    starts = ([(Mb - i - 1, Nb - 1) for i in range(Mb)]
+              + [(0, Nb - i - 1) for i in range(Nb)])
+    idx = np.zeros((Mb, Nb), dtype=np.int32)
+    for w, (jm, jn) in enumerate(starts):
+        while 0 <= jm < Mb and 0 <= jn < Nb:
+            idx[jm, jn] = w
+            jm += 1
+            jn -= 1
+    return idx
+
+
+def apply_wave_gamma(Wr, gamma, td=16):
+    """Multiply each td x td tile of Wr [m, n] by gamma[wave(tile)]."""
+    m, n = Wr.shape
+    Mb, Nb = m // td, n // td
+    g = np.asarray(gamma, np.float32)[wave_index_map(Mb, Nb)]
+    return np.ascontiguousarray(
+        (Wr.reshape(Mb, td, Nb, td) * g[:, None, :, None]).reshape(m, n))
+
+
+CB_V3T = dict(K=1.5, L=16, V=8, tlut_bits=15, decode_mode="quantlut_sym",
+              td_x=16, td_y=16)
+
+
+def decode_expert_v3t(t, tlut, proj, e, m0, n0, cb_params=None, table=None):
+    """Decode ONE expert's projection from a chunked-container layer dict."""
+    cbp = cb_params or CB_V3T
+    c0, off = (e // 32) * 32, e % 32
+    if table is None:
+        table = _np_full_lut_cached(tlut, cbp["L"], cbp["tlut_bits"])
+    m, n = padto(m0), padto(n0)
+    tr = np.asarray(t[f"e{c0}.{proj}.trellis"][off])
+    su = np.asarray(t[f"e{c0}.{proj}.su"][off], np.float32)
+    sv = np.asarray(t[f"e{c0}.{proj}.sv"][off], np.float32)
+    states = _np_unpack_trellis(tr, cbp["td_x"] * cbp["td_y"],
+                                cbp["L"], cbp["K"], cbp["V"])
+    unit = _np_recons(states, table, m, n, cbp["td_x"], cbp["td_y"])
+    unit = unit.astype(np.float16).astype(np.float32)
+    gk = f"e{c0}.{proj}.wave_gamma"
+    if gk in t:
+        unit = apply_wave_gamma(unit, np.asarray(t[gk][off], np.float32),
+                                td=cbp["td_x"])
+    rowside = _np_hadamard(unit) * su
+    colside = _np_hadamard(rowside.T) * sv
+    return np.ascontiguousarray(colside.T[:m0, :n0])
+
+
+def decode_expert_layer_v3t(packed_dir, layer):
+    """Whole-layer reassembly for the chunked container -> gate_up/down fp32."""
+    path = os.path.join(packed_dir, "experts", f"L{layer:02d}.safetensors")
+    t, meta = _read_safetensors_np(path)
+    assert "wave_gamma" in (meta.get("fields") or ""),         "not a chunked-container shard (fields metadata lacks wave_gamma)"
+    tlut, _ = _read_safetensors_np(os.path.join(packed_dir, "experts",
+                                                "codebook.safetensors"))
+    tlut = tlut["tlut"]
+    table = _np_full_lut_cached(tlut, CB_V3T["L"], CB_V3T["tlut_bits"])
+    gate_up = np.empty((NEXP, 2 * INTER, HIDDEN), np.float32)
+    down = np.empty((NEXP, HIDDEN, INTER), np.float32)
+    for e in range(NEXP):
+        gate_up[e, :INTER] = decode_expert_v3t(t, tlut, "gate", e, INTER, HIDDEN,
+                                               table=table)
+        gate_up[e, INTER:] = decode_expert_v3t(t, tlut, "up", e, INTER, HIDDEN,
+                                               table=table)
+        down[e] = decode_expert_v3t(t, tlut, "down", e, HIDDEN, INTER, table=table)
+    return {"gate_up_proj": gate_up, "down_proj": down}

extras.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:013fe06943020ebffb1c3326a38e5dbe789eaef1d2c7f11975793ee91e2da7da
+size 52335992

generation_config.json

@@ -0,0 +1,12 @@
+{
+    "bos_token_id": 248044,
+    "do_sample": true,
+    "eos_token_id": [
+        248046,
+        248044
+    ],
+    "pad_token_id": 248044,
+    "temperature": 1.0,
+    "top_k": 20,
+    "top_p": 0.95
+}

goldens/L0_e0_fp32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d740993d72430e46b44faba103ee2ae0b91d1271c515ad4b21ac4362e790a325
+size 12583352

packed/experts/L00.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46ce93394c8c50deacc5447f022e13618de382cf04f7b0963c74e2beb4e9b1f2
+size 155181208

packed/experts/L01.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0c42283c64130e30c63c2239b1e371dbc42973d7e0598707a7aba692b7bcf6e
+size 155181208

packed/experts/L02.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36ce3634b9da230fba4a9272aa3c8e38ad6fefadfc3092e3e0e110bf2a31402c
+size 155181208

packed/experts/L03.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5faed106e9eca8b533adad896d8d5096d399f5419655b83c88993113e3cc90c
+size 155181208

packed/experts/L04.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25ed7547ce623868f5abe55fefd7e993f3d06627b869afbb5d2ac2e59ce27cd8
+size 155181208

packed/experts/L05.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7263de6c512148a8428d69b8ee9e9863024668dd7c7387d0b172a946b7058bbc
+size 155181208

packed/experts/L06.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f72aa6d5af5346bf0b481354b7bb25b2f0ed74633b0fdb57b3de04afe281a8c
+size 155181208

packed/experts/L07.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6dc718b28d8d33f5d3582bdc96d681db4a7cae62593a2267dc2d84f85b312c5
+size 155181208

packed/experts/L08.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44989b42d4c89c9182e3e0868fd8ca2cbcecc48dccb3e7c167698c9d0e413b00
+size 155181208

packed/experts/L09.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b35db17e3ea832dbcd69b663250759cd8769e8a427ef1c0b670f6c897e11eb7b
+size 155181208

packed/experts/L10.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:216d9e27b624d6b3ba18196ab82be7b4c7ad9f3c8c5593ff90d62c9d741b700b
+size 155181208

packed/experts/L11.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0973ce67cb7eded3042b17a1ef4e9259ae036284090b9b4800f44e2bcfe03441
+size 155181208

packed/experts/L12.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41c8d788406abe62201ce7b7a8826bc79126eac8f609eaa9a5a89f8dfd545331
+size 155181208

packed/experts/L13.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd39c71803a28607e5dac60188bc084535cc651c519ac2dd023f681cd1544fd4
+size 155181208

packed/experts/L14.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f43f300d69327861bba48a068e5fe2578340280b84d2868b0e7ceceafaa4b816
+size 155181208

packed/experts/L15.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2cc4710dfb5d7926f5c1ccba606a187cf2dc5fd15b1d29cf32f5d092817786f
+size 155181208

packed/experts/L16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d70daff68a440a13107d6fc6f3ac5c58d79f845d0da46110ecbd8f1e811520d9
+size 155181208

packed/experts/L17.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27d382c46bf01a0e9d04f014b66b98dbfe2bfff10145e03552db0b679fc761f7
+size 155181208

packed/experts/L18.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82705f3a85d149e695323592dc7a8df0e0fd856b83f7b3bad0a6cbb93999d446
+size 155181208

packed/experts/L19.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3004bbdd61cd7491c66b64706925611b4af2202917f3c54e29ed1ec4c7675af2
+size 155181208

packed/experts/L20.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a65552b5c5c97b1b306308c1d7eaa3b04a9327f5b3304af01296f0da89d14f6
+size 155181208

packed/experts/L21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7152fa5e1c856fd51d8986dc0c615eb8aaa1779e8361cf2db20de7eb544919c2
+size 155181208

packed/experts/L22.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c2a4ef0896c2ef1fd768e0d6e42baf351e29740133e34e2651baa2790288ff9
+size 155181208

packed/experts/L23.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e8218c7e6cb3a841bd69a33a3577ea9b2330cd183bc8051245af13d5db79526
+size 155181208

packed/experts/L24.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85598864d8a0c95b52bddba92daf808cf0982ce04a93f04c96641b11724553c5
+size 155181208

packed/experts/L25.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34a32d1cf323a5b6f0bec152ca30e1922806e6da87383ae6733730f43913503e
+size 155181208

packed/experts/L26.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ac4e7cf8efcc94fa72848e337c86b1f6594cefe8e18006800c87dfd33334e30
+size 155181208

packed/experts/L27.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ed5ed0e4231a394271f3b20700091ceaa422d661e685722dc0a047635fb129d
+size 155181208

packed/experts/L28.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44d34c83d6384b00de68cd65b4e7416f8b899a63df6986d1d6ea1c6209f40ca1
+size 155181208

packed/experts/L29.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f86acf7df3a4fb2188c55aecfc1f23c16e16df258e885af8b066175052c8510
+size 155181208

packed/experts/L30.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22f204b0f6924402e48fd5553ffa88b225b659d723a4b0079dba1296731a09e0
+size 155181208

packed/experts/L31.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04ce1c0913d0870e3f3ef3e713ff388481494ea790f1166613e8b3d340de84a5
+size 155181208

packed/experts/L32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:735ee4c7e4c578d0a0d78180220052eb7e5a6bc15b697adebd526881770c23f6
+size 155181208

packed/experts/L33.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:269933ec55cde295936c06e4a657cd691b39b1d15a6cecbdceeaed2644e0a9c4
+size 155181208

packed/experts/L34.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a87eb2a8df870c181047addf0d219dea5445c94b1dcd250550ed8cdaad0f8fd5
+size 155181208

packed/experts/L35.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91bf39bcbc9dd7e060c7f9582b8a5498001d23f46a785adcea4862a71b7eac88
+size 155181208

packed/experts/L36.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60b4103e563fb55053eaeafd9b45a6cb00afbbe966da2537f159d3d218c3699d
+size 155181208