Initial upload (MLX artifact for IEEE Cloud Summit 2026 paper)

README.md

@@ -0,0 +1,99 @@
+---
+library_name: mlx
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+tags:
+- mistral-common
+- mlx
+- quantization
+- bias-evaluation
+---
+
+# mistral-7b-instruct-v0.3-bf16 (MLX, CBA artifact)
+
+MLX-format BF16 (uncompressed baseline) variant of [`mistralai/Mistral-7B-Instruct-v0.3`](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3).
+
+This is one of the **15 model artifacts** from the paper:
+
+> **Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels**
+> Plawan Kumar Rath, Rahul Maliakkal. *IEEE Cloud Summit 2026*.
+> Code: <https://github.com/plawanrath/compression-bias-amplification>
+
+## Quantization
+
+This is the **BF16 baseline** used as the uncompressed reference in the paper. Weights have been re-serialized via `mlx_lm.convert` (no quantization) so this directory is loadable directly by MLX without an extra conversion step.
+
+## How this artifact was produced
+
+```bash
+python -m mlx_lm.convert \
+    --hf-path mistralai/Mistral-7B-Instruct-v0.3 \
+    --mlx-path ./mistral-7b-instruct-v0.3-bf16 \
+
+```
+
+This is the **exact** artifact used to produce the inference results in §4.3 of the paper (911,100 records over BBQ ambiguous, 5 seeds × 12,148 items × 15 configs).
+
+## Usage (MLX)
+
+```bash
+pip install mlx-lm
+```
+
+```python
+from mlx_lm import load, generate
+
+model, tokenizer = load("plawanrath/mistral-7b-instruct-v0.3-bf16-mlx-cba")
+prompt = tokenizer.apply_chat_template(
+    [{"role": "user", "content": "Hello!"}],
+    add_generation_prompt=True,
+    tokenize=False,
+)
+print(generate(model, tokenizer, prompt=prompt, max_tokens=128))
+```
+
+Or via CLI:
+
+```bash
+mlx_lm.generate --model plawanrath/mistral-7b-instruct-v0.3-bf16-mlx-cba --prompt "Hello!"
+```
+
+## Paper findings relevant to this variant
+
+The paper documents a **dose-response** relationship between quantization aggressiveness and emergent stereotypical behavior on BBQ ambiguous questions:
+
+| Variant | % of BF16-unbiased items that became biased |
+|---|---|
+| Q8 | 0.1–0.9% |
+| Q6 | 0.3–1.3% |
+| Q4 | 2.2–5.6% |
+| Q3 | 6.0–21.1% |
+
+These changes are largely **invisible to perplexity** (<0.5% shift at Q8, <3% at Q4 across all three families). Treat any deployment of compressed instruction-tuned models on fairness-sensitive tasks accordingly.
+
+## Model details
+
+- **Base model:** [`mistralai/Mistral-7B-Instruct-v0.3`](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3)
+- **Family:** Mistral
+- **Parameters:** 7.2B
+- **Precision:** BF16 (uncompressed baseline)
+- **Format:** MLX (Apple Silicon)
+- **Conversion framework:** [`mlx-lm`](https://github.com/ml-explore/mlx-lm)
+
+## License
+
+Inherited from the base model (`apache-2.0`). See the upstream model page for the full license text.
+
+## Citation
+
+```bibtex
+@inproceedings{rath2026quantization,
+  title     = { Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels },
+  author    = {Rath, Plawan Kumar and Maliakkal, Rahul},
+  booktitle = { IEEE Cloud Summit 2026 },
+  year      = {2026}
+}
+```

chat_template.jinja

@@ -0,0 +1,87 @@
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
+
+{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
+{%- set ns = namespace() %}
+{%- set ns.index = 0 %}
+{%- for message in loop_messages %}
+    {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
+            {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if message["role"] == "user" %}
+        {%- if tools is not none and (message == user_messages[-1]) %}
+            {{- "[AVAILABLE_TOOLS] [" }}
+            {%- for tool in tools %}
+                {%- set tool = tool.function %}
+                {{- '{"type": "function", "function": {' }}
+                {%- for key, val in tool.items() if key != "return" %}
+                    {%- if val is string %}
+                        {{- '"' + key + '": "' + val + '"' }}
+                    {%- else %}
+                        {{- '"' + key + '": ' + val|tojson }}
+                    {%- endif %}
+                    {%- if not loop.last %}
+                        {{- ", " }}
+                    {%- endif %}
+                {%- endfor %}
+                {{- "}}" }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- else %}
+                    {{- "]" }}
+                {%- endif %}
+            {%- endfor %}
+            {{- "[/AVAILABLE_TOOLS]" }}
+            {%- endif %}
+        {%- if loop.last and system_message is defined %}
+            {{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
+        {%- else %}
+            {{- "[INST] " + message["content"] + "[/INST]" }}
+        {%- endif %}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "[TOOL_CALLS] [" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {{- out[:-1] }}
+            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
+                {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+            {%- endif %}
+            {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]" + eos_token }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif message["role"] == "assistant" %}
+        {{- " " + message["content"]|trim + eos_token}}
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
+        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
+            {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+        {%- endif %}
+        {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
+    {%- else %}
+        {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
+    {%- endif %}
+{%- endfor %}

config.json

@@ -0,0 +1,25 @@
+{
+    "architectures": [
+        "MistralForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 14336,
+    "max_position_embeddings": 32768,
+    "model_type": "mistral",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 32,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 1000000.0,
+    "sliding_window": null,
+    "tie_word_embeddings": false,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.42.0.dev0",
+    "use_cache": true,
+    "vocab_size": 32768
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.42.0.dev0"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bff1097a50c91877a325700af3e0b239517c120ea5844cc0ee3423ff3a786a22
+size 5268238200

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f851a5895828ed5a1aa9b3895fb118050c299b491169770edf89540daa53cc8
+size 5352141224

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ef3a5fccd0539ca2f1515f57e71b38f7aa4051f43c1e492ed1f68385592e3ad
+size 3875701486

model.safetensors.index.json

@@ -0,0 +1,299 @@
+{
+    "metadata": {
+        "total_size": 14496047104,
+        "total_parameters": 7248023552
+    },
+    "weight_map": {
+        "lm_head.weight": "model-00003-of-00003.safetensors",
+        "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.11.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.11.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.23.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+        "model.layers.24.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+        "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+        "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+        "model.norm.weight": "model-00003-of-00003.safetensors"
+    }
+}

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": true,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "tool_parser_type": "mistral",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}