Instructions to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AX-Qwen3.6-27B-MLX-AXQ-6bit / model-manifest.json

AutomatosX

Publish no-MTP sibling of certified MTP pack (language path identical; mtp.safetensors omitted).

5e6421e verified about 1 month ago

Raw

History Blame Contribute Delete

433 kB

	{
	"schema_version": "ax.native_model.v1",
	"model_family": "qwen3_5",
	"tensor_format": "safetensors",
	"layer_count": 64,
	"hidden_size": 5120,
	"intermediate_size": 17408,
	"attention_head_count": 24,
	"attention_head_dim": 256,
	"kv_head_count": 4,
	"vocab_size": 248320,
	"tie_word_embeddings": false,
	"rope_theta": 10000000,
	"no_rope_layer_interval": 0,
	"intermediate_size_mlp": 0,
	"attn_output_gate": true,
	"partial_rotary_factor": 0.25,
	"rms_norm_eps": 0.000001,
	"moe_norm_topk_prob": true,
	"hidden_size_per_layer_input": 0,
	"linear_attention": {
	"full_attention_interval": 4,
	"num_value_heads": 48,
	"num_key_heads": 16,
	"key_head_dim": 128,
	"value_head_dim": 128,
	"conv_kernel_dim": 4
	},
	"think_start_token_id": 248068,
	"think_end_token_id": 248069,
	"dropped_tensors": {
	"count": 989,
	"media_role_hits": 0,
	"names_sample": [
	"language_model.model.embed_tokens.biases",
	"language_model.model.embed_tokens.scales",
	"language_model.model.layers.0.linear_attn.in_proj_a.biases",
	"language_model.model.layers.0.linear_attn.in_proj_a.scales",
	"language_model.model.layers.0.linear_attn.in_proj_b.biases",
	"language_model.model.layers.0.linear_attn.in_proj_b.scales",
	"language_model.model.layers.0.linear_attn.in_proj_qkv.biases",
	"language_model.model.layers.0.linear_attn.in_proj_qkv.scales",
	"language_model.model.layers.0.linear_attn.in_proj_z.biases",
	"language_model.model.layers.0.linear_attn.in_proj_z.scales",
	"language_model.model.layers.0.mlp.down_proj.biases",
	"language_model.model.layers.0.mlp.down_proj.scales",
	"language_model.model.layers.0.mlp.gate_proj.biases",
	"language_model.model.layers.0.mlp.gate_proj.scales",
	"language_model.model.layers.0.mlp.up_proj.biases",
	"language_model.model.layers.0.mlp.up_proj.scales"
	]
	},
	"tensors": [
	{
	"name": "language_model.model.norm.weight",
	"role": "final_norm",
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2816435755,
	"length_bytes": 10240
	},
	{
	"name": "language_model.lm_head.weight",
	"role": "lm_head",
	"dtype": "bf16",
	"shape": [
	248320,
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 4651,
	"length_bytes": 2542796800
	},
	{
	"name": "model.visual.blocks.0.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 37128,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 39432,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.0.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 2693640,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.0.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 2700552,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 10663176,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 10671784,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 20588200,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 20590504,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.0.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30506920,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30509224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30511528,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.0.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30513832,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30516136,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 30518440,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.1.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 33172648,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.1.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 33179560,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 41142184,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 41150792,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 51067208,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 51069512,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.1.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60985928,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60988232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60990536,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.1.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60992840,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60995144,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 60997448,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.10.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 63651656,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.10.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 63658568,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 71621192,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 71629800,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 81546216,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 81548520,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.10.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91464936,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91467240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91469544,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.10.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91471848,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91474152,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 91476456,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.11.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 94130664,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.11.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 94137576,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 102100200,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 102108808,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 112025224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 112027528,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.11.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121943944,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121946248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121948552,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.11.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121950856,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121953160,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 121955464,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.12.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 124609672,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.12.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 124616584,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 132579208,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 132587816,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 142504232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 142506536,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.12.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152422952,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152425256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152427560,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.12.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152429864,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152432168,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 152434472,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.13.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 155088680,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.13.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 155095592,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 163058216,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 163066824,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 172983240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 172985544,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.13.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182901960,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182904264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182906568,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.13.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182908872,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182911176,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 182913480,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.14.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 185567688,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.14.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 185574600,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 193537224,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 193545832,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 203462248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 203464552,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.14.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213380968,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213383272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213385576,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.14.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213387880,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213390184,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 213392488,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.15.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 216046696,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.15.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 216053608,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 224016232,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 224024840,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 233941256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 233943560,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.15.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243859976,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243862280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243864584,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.15.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243866888,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243869192,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 243871496,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.16.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 246525704,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.16.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 246532616,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 254495240,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 254503848,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 264420264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 264422568,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.16.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274338984,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274341288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274343592,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.16.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274345896,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274348200,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 274350504,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.17.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 277004712,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.17.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 277011624,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 284974248,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 284982856,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 294899272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 294901576,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.17.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304817992,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304820296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304822600,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.17.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304824904,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304827208,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 304829512,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.18.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 307483720,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.18.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 307490632,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 315453256,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 315461864,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 325378280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 325380584,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.18.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335297000,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335299304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335301608,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.18.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335303912,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335306216,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 335308520,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.19.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 337962728,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.19.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 337969640,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 345932264,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 345940872,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 355857288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 355859592,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.19.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365776008,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365778312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365780616,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.19.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365782920,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365785224,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 365787528,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.2.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 368441736,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.2.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 368448648,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 376411272,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 376419880,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 386336296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 386338600,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.2.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396255016,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396257320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396259624,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.2.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396261928,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396264232,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 396266536,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.20.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 398920744,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.20.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 398927656,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 406890280,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 406898888,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 416815304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 416817608,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.20.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426734024,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426736328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426738632,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.20.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426740936,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426743240,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 426745544,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.21.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 429399752,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.21.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 429406664,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 437369288,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 437377896,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 447294312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 447296616,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.21.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457213032,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457215336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457217640,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.21.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457219944,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457222248,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 457224552,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.22.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 459878760,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.22.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 459885672,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 467848296,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 467856904,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 477773320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 477775624,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.22.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487692040,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487694344,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487696648,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.22.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487698952,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487701256,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 487703560,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.23.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 490357768,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.23.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 490364680,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 498327304,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 498335912,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 508252328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 508254632,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.23.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518171048,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518173352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518175656,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.23.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518177960,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518180264,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 518182568,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.24.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 520836776,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.24.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 520843688,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 528806312,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 528814920,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 538731336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 538733640,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.24.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548650056,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548652360,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548654664,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.24.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548656968,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548659272,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 548661576,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.25.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 551315784,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.25.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 551322696,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 559285320,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 559293928,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 569210344,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 569212648,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.25.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579129064,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579131368,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579133672,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.25.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579135976,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579138280,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 579140584,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.26.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 581794792,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.26.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 581801704,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 589764328,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 589772936,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 599689352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 599691656,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.26.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609608072,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609610376,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609612680,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.26.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609614984,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609617288,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 609619592,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.3.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 612273800,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.3.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 612280712,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 620243336,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 620251944,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 630168360,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 630170664,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.3.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640087080,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640089384,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640091688,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.3.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640093992,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640096296,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 640098600,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.4.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 642752808,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.4.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 642759720,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 650722344,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 650730952,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 660647368,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 660649672,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.4.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670566088,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670568392,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670570696,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.4.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670573000,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670575304,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 670577608,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.5.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 673231816,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.5.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 673238728,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 681201352,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 681209960,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 691126376,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 691128680,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.5.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701045096,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701047400,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701049704,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.5.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701052008,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701054312,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 701056616,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.6.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 703710824,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.6.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 703717736,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 711680360,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 711688968,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 721605384,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 721607688,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.6.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731524104,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731526408,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731528712,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.6.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731531016,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731533320,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 731535624,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.7.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 734189832,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.7.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 734196744,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 742159368,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 742167976,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 752084392,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 752086696,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.7.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762003112,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762005416,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762007720,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.7.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762010024,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762012328,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 762014632,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.8.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 764668840,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.8.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 764675752,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 772638376,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 772646984,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 782563400,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 782565704,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.8.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792482120,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792484424,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792486728,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.8.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792489032,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.attn.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792491336,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.attn.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 792493640,
	"length_bytes": 2654208
	},
	{
	"name": "model.visual.blocks.9.attn.qkv.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456
	],
	"file": "vision.safetensors",
	"offset_bytes": 795147848,
	"length_bytes": 6912
	},
	{
	"name": "model.visual.blocks.9.attn.qkv.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	3456,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 795154760,
	"length_bytes": 7962624
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 803117384,
	"length_bytes": 8608
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 803125992,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 813042408,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.mlp.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	4304
	],
	"file": "vision.safetensors",
	"offset_bytes": 813044712,
	"length_bytes": 9916416
	},
	{
	"name": "model.visual.blocks.9.norm1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822961128,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822963432,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822965736,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.blocks.9.norm2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 822968040,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.merger.linear_fc1.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 822970344,
	"length_bytes": 9216
	},
	{
	"name": "model.visual.merger.linear_fc1.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	4608,
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 822979560,
	"length_bytes": 42467328
	},
	{
	"name": "model.visual.merger.linear_fc2.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "vision.safetensors",
	"offset_bytes": 865446888,
	"length_bytes": 10240
	},
	{
	"name": "model.visual.merger.linear_fc2.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	5120,
	4608
	],
	"file": "vision.safetensors",
	"offset_bytes": 865457128,
	"length_bytes": 47185920
	},
	{
	"name": "model.visual.merger.norm.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912643048,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.merger.norm.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912645352,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.patch_embed.proj.bias",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 912647656,
	"length_bytes": 2304
	},
	{
	"name": "model.visual.patch_embed.proj.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	1152,
	3,
	2,
	16,
	16
	],
	"file": "vision.safetensors",
	"offset_bytes": 912649960,
	"length_bytes": 3538944
	},
	{
	"name": "model.visual.pos_embed.weight",
	"role": "other",
	"dtype": "bf16",
	"shape": [
	2304,
	1152
	],
	"file": "vision.safetensors",
	"offset_bytes": 916188904,
	"length_bytes": 5308416
	},
	{
	"name": "language_model.model.embed_tokens.weight",
	"role": "token_embedding",
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 8
	},
	"shape": [
	248320,
	1280
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3232005736,
	"length_bytes": 1271398400
	},
	{
	"name": "language_model.model.layers.0.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4966700328,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.0.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2108008904,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.0.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4740352808,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5227685256,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2260941000,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.0.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1532596904,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.0.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4913851688,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.0.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2858836200,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5274871368,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 691673384,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4887637288,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.0.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 0,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5020647720,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.0.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3224442984,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.0.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 0,
	"dtype": "bf16",
	"shape": [
	5120,
	6144
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4568469096,
	"length_bytes": 62914560
	},
	{
	"name": "language_model.model.layers.1.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 914388360,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.1.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4940311848,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.1.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3123691624,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2677372296,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1948899688,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.1.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4639903336,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.1.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5080956296,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.1.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2861621576,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4882919208,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1903577448,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2078998984,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.1.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2063270344,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.1.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 1,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2473479976,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.1.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 1,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4940322088,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.2.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4503404136,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.2.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4503414376,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.2.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2632800136,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2015920584,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4507192936,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.2.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5274871272,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.2.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2913730568,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.2.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4672359112,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5275066024,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1945008488,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	10240,
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1610912520,
	"length_bytes": 104857600
	},
	{
	"name": "language_model.model.layers.2.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4784917288,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.2.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 2,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2755278312,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.2.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 2,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4551757416,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.3.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3216112744,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.3.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2574771848,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.3.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2473469640,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.3.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3092070504,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.3.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 915399656,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.3.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2421348040,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.3.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 3,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5020647208,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.3.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1558811400,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.3.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2331228360,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.3.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2814271720,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.3.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 3,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3171548264,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3231987560,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.4.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2190553800,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.4.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 931153992,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2758063848,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5036391816,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.4.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2473479880,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.4.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2752411016,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.4.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2489216552,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 392731240,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2574772360,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2726196616,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.4.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2574956680,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.4.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 4,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1158026408,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.4.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 4,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2454771400,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.5.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2105213384,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.5.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2000181704,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.5.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5132002696,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 740963624,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1834728808,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.5.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3092070408,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.5.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2260859080,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.5.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5036384040,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2127516360,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 515652200,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2544133768,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.5.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2473480232,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.5.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 5,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2110804424,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.5.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 5,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2192202440,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.6.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5275055784,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.6.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1718555400,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.6.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5183120776,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2210880200,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4970501928,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.6.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5273888136,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.6.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 523055720,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.6.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1830960392,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1896660328,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1903700328,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1905686888,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.6.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3107962984,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.6.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 6,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1830960136,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.6.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 6,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2110804680,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.7.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1561760520,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.7.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1428531880,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.7.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1382973864,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.7.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1516868264,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.7.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1382963624,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.7.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1573884680,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.7.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 7,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1432318632,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.7.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2723575176,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.7.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1335606184,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.7.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4808528168,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.7.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 7,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1288174504,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2805413608,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.8.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1210250664,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.8.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2861629352,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.8.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1059379272,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.8.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1449038504,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.8.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1126233672,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.8.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5275188904,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.8.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1993464168,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 793223560,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1164580264,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1217306024,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.8.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2000191944,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.8.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 8,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1830959880,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.8.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 8,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2309929160,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.9.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 914398600,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.9.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5020636968,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.9.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 864253320,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.9.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2376783560,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.9.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5275270824,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.9.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 915391880,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.9.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1288092584,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.9.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 915409896,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3224312424,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1053624392,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1532597000,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.9.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 915425352,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.9.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 9,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2590693000,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.9.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 9,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1432319144,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.10.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 790297480,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.10.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 788313384,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.10.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1764113160,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.10.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 570461896,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.10.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1718565640,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.10.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5275055688,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.10.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 864171400,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.10.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 790289704,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 790307720,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 793354120,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 837957000,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.10.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 802068360,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.10.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 10,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 3231997800,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.10.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 10,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 820589960,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.11.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2490215048,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.11.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2811485928,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.11.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 670854952,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.11.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 670865704,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.11.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1432300712,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.11.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 701150504,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.11.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 11,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 670865192,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.11.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 691796264,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.11.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1165686184,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.11.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 982927432,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.11.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 11,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 617934632,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 523045480,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.12.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 801075080,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.12.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 471087720,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 525897416,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1382984360,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.12.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2752492936,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.12.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1332738984,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.12.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 524120680,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2473285320,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 524128456,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1126233768,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.12.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4656630472,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.12.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 12,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1382984104,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.12.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 12,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1879293288,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.13.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 376992104,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.13.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5092179336,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.13.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 326857064,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2499405448,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 277868904,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.13.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 617811656,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.13.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 5131920776,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.13.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2907176872,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	48,
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 392854120,
	"length_bytes": 491520
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 617811752,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 402356840,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.13.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4913933608,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.13.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 13,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 392730984,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.13.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 13,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 377002344,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.14.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 207419816,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.14.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 207409576,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.14.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 162844584,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.14.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4673342248,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.14.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 90427304,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.14.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 270344872,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.14.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1050757192,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.14.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 270344968,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 270352744,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 270537064,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 14,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2593478536,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.14.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	6144,
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2127639240,
	"length_bytes": 62914560
	},
	{
	"name": "language_model.model.layers.14.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 270344616,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.14.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 14,
	"dtype": "bf16",
	"shape": [
	5120,
	6144
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 207430056,
	"length_bytes": 62914560
	},
	{
	"name": "language_model.model.layers.15.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 51597224,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.15.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 207409064,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.15.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4808517928,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.15.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 4639903432,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.15.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 47490472,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.15.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 56184744,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.15.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 47500712,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.15.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 48811944,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.15.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 1243520424,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.15.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 15,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2925992,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.15.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 15,
	"dtype": "bf16",
	"shape": [
	17408,
	5120
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 2913812488,
	"length_bytes": 178257920
	},
	{
	"name": "language_model.model.layers.16.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5092084130,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.16.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849762434,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.16.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5041818338,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3881096450,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1376900354,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.16.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5086382914,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.16.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00001-of-00004.safetensors",
	"offset_bytes": 58792,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.16.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5325300386,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5178089890,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4463324002,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4563512610,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.16.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3072626082,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.16.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 16,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5289818530,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.16.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 16,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3735967138,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.17.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1922678754,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.17.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3995300610,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.17.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4723755810,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4769311010,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5130115490,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.17.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5328085762,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.17.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5292440226,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.17.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4952343586,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5177967010,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2403076354,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5002806498,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.17.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4927112226,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.17.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 17,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3428850082,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.17.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 17,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3279576482,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.18.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4362897666,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.18.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5176318370,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.18.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4250740994,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4368478466,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4957258882,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.18.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2659763554,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.18.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3063205282,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.18.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4415846146,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4508994498,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5089168290,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4602834210,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.18.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1808289826,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.18.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 18,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4347168770,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.18.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 18,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3092123042,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.19.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5038213858,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.19.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1923687394,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.19.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4019692290,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.19.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4085566722,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.19.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4013128450,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.19.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2456798306,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.19.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 19,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4038216450,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.19.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5289818786,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.19.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2085009058,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.19.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1876475778,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.19.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 19,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3928453890,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.20.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3751713794,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.20.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3428850338,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.20.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3331365282,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4629048610,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4197984514,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.20.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1232413858,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.20.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3829896194,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.20.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3061223746,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4295305474,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4463139682,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1692815906,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.20.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4111133698,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.20.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 20,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4553681858,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.20.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 20,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2900819074,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.21.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1422463522,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.21.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849772674,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.21.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3554890658,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.21.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1544857122,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.21.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3488043938,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.21.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3023061634,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.21.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3297033122,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.21.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3027485410,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3829711874,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849639042,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3029451586,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.21.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3212873122,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.21.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 21,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2543157602,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.21.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 21,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2918193794,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.22.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3055665986,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.22.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3735941538,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.22.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4509117378,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.22.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2610437218,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.22.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5178274210,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.22.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2662548930,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.22.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849782914,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.22.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2010031586,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2754463266,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2799281186,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	10240,
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3108015522,
	"length_bytes": 104857600
	},
	{
	"name": "language_model.model.layers.22.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4022487810,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.22.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 22,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2659753058,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.22.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 22,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1656360994,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.23.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5030011618,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.23.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3437380514,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.23.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4460410722,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.23.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1336585474,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.23.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5325290146,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.23.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3299908002,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.23.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 23,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3440002466,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.23.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2545287778,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.23.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2978497154,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.23.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2804992546,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.23.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 23,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4142107394,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2324893954,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.24.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1352314114,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.24.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2326071554,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2271973634,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2218070274,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.24.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1421464834,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.24.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849557026,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.24.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2403199234,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3055686466,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4344260610,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2425013346,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.24.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2405999970,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.24.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 24,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3432628898,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.24.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 24,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2371619074,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.25.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2012980898,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.25.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2659753314,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.25.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2933932674,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3378715042,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2707113506,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.25.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4464429922,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.25.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4463042402,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.25.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2212484258,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2076502178,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1251910914,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2186269858,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.25.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2129573538,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.25.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 25,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2493826146,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.25.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 25,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2031668898,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.26.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1826813986,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.26.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1826803746,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.26.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3235012002,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4038216962,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1760940066,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.26.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1876475682,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.26.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2076397218,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.26.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1921040258,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5289695650,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3228601762,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	10240,
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4813875490,
	"length_bytes": 104857600
	},
	{
	"name": "language_model.model.layers.26.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2014957218,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.26.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 26,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1829125666,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.26.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 26,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2145302178,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.27.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2894429314,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.27.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1650789922,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.27.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1650779682,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.27.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1676923426,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.27.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3055676226,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.27.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1720996386,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.27.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 27,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1672089634,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.27.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3055809346,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.27.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849864834,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.27.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1597859362,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.27.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 27,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1497507362,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1335592194,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.28.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2799404066,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.28.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2497602402,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.28.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1265960194,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.28.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3669094818,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.28.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1421464930,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.28.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1828060706,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.28.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5001823362,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1422481442,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1422604322,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1423779362,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.28.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3997399810,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.28.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 28,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3878310914,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.28.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 28,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1353307394,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.29.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1206025378,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.29.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1206015138,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.29.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2662549026,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3440366498,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1155872418,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.29.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1421465026,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.29.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3295305122,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.29.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2012980802,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2079471778,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1827930146,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5295307426,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.29.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1236182274,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.29.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 29,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2423367010,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.29.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 29,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1208820898,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.30.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1085331170,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.30.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5130105250,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.30.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1924670946,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.30.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1040766690,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.30.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 985045730,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.30.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1103036482,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.30.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1155790498,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.30.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3751706018,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1103044258,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3440182178,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1129568418,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.30.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1110562978,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.30.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 30,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1103036226,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.30.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 30,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1087307586,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.31.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 972921570,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.31.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849761922,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.31.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 967985122,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.31.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2168902818,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.31.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 967967202,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.31.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5095862690,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.31.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 31,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 967995362,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.31.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 969142754,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.31.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3753362434,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.31.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 923402722,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.31.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 31,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 870482402,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 743345026,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.32.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4415835906,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.32.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 698780546,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4673613090,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 648645506,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.32.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1087307490,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.32.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1672090146,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.32.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 759083906,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 759091682,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 759276002,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 825590242,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.32.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	6144,
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 759398882,
	"length_bytes": 62914560
	},
	{
	"name": "language_model.model.layers.32.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 32,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4336723714,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.32.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 32,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 743355266,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.33.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 602826434,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.33.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5038203618,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.33.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 558261954,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2754716706,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1829125922,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.33.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 618565314,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.33.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1597777442,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.33.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 618565410,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 618573186,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 618703746,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 622431106,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.33.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1994302946,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.33.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 33,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4101295362,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.33.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 33,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 602836674,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.34.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 527556962,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.34.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 527546722,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.34.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 480196962,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3829978114,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1449993762,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.34.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 528550498,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.34.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 558180034,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.34.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5086382818,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 528558274,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3996293890,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 531965634,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.34.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2387347714,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.34.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 34,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 528550242,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.34.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 34,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4347169026,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.35.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4560891170,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.35.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 384997218,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.35.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 384986978,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.35.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 385981282,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.35.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5289685410,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.35.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 448739682,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.35.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 35,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 384997730,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.35.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3437381026,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.35.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 315354978,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.35.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3599455138,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.35.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 35,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 401709922,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 266758306,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.36.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3751695778,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.36.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 5245120930,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 222193826,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4464430018,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.36.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 309702402,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.36.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 309702498,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.36.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4556467394,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1758031906,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 1827807266,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4295428354,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.36.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 284471042,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.36.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 36,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 972921314,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.36.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 36,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 268734722,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.37.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2933922434,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.37.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 972911074,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.37.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 76934210,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 124283970,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4415846242,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.37.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 194407490,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.37.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 213756066,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.37.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2849638946,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2324904194,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2754586146,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2047397538,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.37.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 196389026,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.37.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 37,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4134726658,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.37.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 37,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 170814530,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.38.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2880994,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.38.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2870754,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.38.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5281962540,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2551685218,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2747193708,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.38.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 19602914,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.38.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 2604784738,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.38.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 266768546,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 19603010,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 19733570,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 45149250,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.38.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 19917890,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.38.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 38,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 4927111970,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.38.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 38,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00002-of-00004.safetensors",
	"offset_bytes": 3874274,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.39.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3740195180,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.39.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5142315724,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.39.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4594999468,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.39.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5145291468,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.39.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4838427596,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.39.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3863449804,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.39.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 39,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4539818060,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.39.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2878240204,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.39.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2419938124,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.39.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5075461068,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.39.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 39,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4957188556,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.40.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3517718956,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.40.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4654025484,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.40.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4539818572,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3427596716,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5228919244,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.40.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2999940236,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.40.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3740113260,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.40.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2880861644,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1363182668,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4585455692,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5029626828,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.40.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4191107468,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.40.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 40,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5221500108,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.40.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 40,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4788251596,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.41.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4485720140,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.41.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5221500364,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.41.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4396591180,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.41.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3695540588,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.41.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1939278284,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.41.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4094275116,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.41.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4597802668,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.41.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4585640012,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4097068172,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4388931660,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4654043500,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.41.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4249772428,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.41.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 41,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3944066764,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.41.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 41,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4772515276,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.42.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4191097228,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.42.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4214700428,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.42.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4049628620,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4146522508,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4002189260,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.42.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4307771884,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.42.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3692509548,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.42.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2997843500,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4539633740,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4310884940,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4346817100,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.42.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5161020108,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.42.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 42,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3519695788,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.42.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 42,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4218315148,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.43.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4314038860,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.43.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2364552012,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.43.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3801169868,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.43.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3406297260,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.43.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3742817228,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.43.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4804062156,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.43.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 43,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3633393420,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.43.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3689888108,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.43.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2188299820,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.43.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3956641740,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.43.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 43,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3749552588,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3672183148,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.44.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3517708716,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.44.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3361732524,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3473144236,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3224083884,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.44.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4680257900,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.44.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1367091788,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.44.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3587052204,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3587052300,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3633393932,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3290953644,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.44.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3674159468,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.44.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 44,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1747633324,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.44.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 44,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3519696044,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.45.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3571313324,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.45.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4592203948,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.45.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2833176428,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.45.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2928211500,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.45.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3317168044,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.45.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5273483724,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.45.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4311990860,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.45.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3015676652,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3015684428,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5075338188,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3176214348,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.45.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3642036588,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.45.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 45,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3427596460,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.45.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 45,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2999940332,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.46.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2697810764,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.46.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1877289996,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.46.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4683050956,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2653246284,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3818885324,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.46.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2741361484,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.46.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2820757868,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.46.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2743327756,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 8
	},
	"shape": [
	48,
	1280
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2744326508,
	"length_bytes": 245760
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2826492268,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2791758188,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.46.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4316660300,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.46.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 46,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4098181772,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.46.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 46,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2719939404,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.47.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2464502604,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.47.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3742816716,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.47.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2364254540,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.47.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3271441324,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.47.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5029606348,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.47.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2608026444,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.47.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 47,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2364387660,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.47.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2371441484,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.47.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2316897100,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.47.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4842861516,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.47.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 47,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4489498700,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3801180108,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.48.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1748616620,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.48.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5176756428,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2143735340,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4912624076,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.48.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2236632876,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.48.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2823625068,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.48.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2266598028,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2239589772,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4887425996,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	10240,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4733185996,
	"length_bytes": 39321600
	},
	{
	"name": "language_model.model.layers.48.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2242513292,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.48.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 48,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2263812492,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.48.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 48,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4292043244,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.49.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1876296716,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.49.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1876286476,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.49.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4101950348,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.49.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1831721996,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.49.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2006125004,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.49.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1926288652,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.49.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4585373772,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.49.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1928910188,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2738453324,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1939093964,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2079167020,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.49.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1475633228,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.49.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 49,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1924649996,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.49.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 49,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3208355148,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.50.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4345659980,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.50.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1931859404,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.50.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1641784908,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2374226764,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3588828940,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.50.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4654043404,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.50.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1747551404,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.50.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1686349388,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1712727724,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1712850604,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1721337004,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.50.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2702572364,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.50.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 50,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2236632620,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.50.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 50,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1625073228,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.51.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2645054284,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.51.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5145283276,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.51.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1382561868,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.51.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3802173388,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.51.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1382551628,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.51.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1537073228,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.51.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 51,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3519695276,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.51.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1620478028,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.51.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4441155660,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.51.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1877300236,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.51.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 51,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2467124044,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1363305548,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.52.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1359404108,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.52.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2515293004,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1314839628,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1264696908,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.52.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2077405644,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.52.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4049546700,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.52.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2316897004,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1367173708,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2999817356,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1449418828,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.52.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5121180364,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.52.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 52,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3818885068,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.52.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 52,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4896895436,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.53.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1227248268,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.53.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4613710604,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.53.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1135334028,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1182683788,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1087984268,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.53.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1228241804,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.53.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1257920076,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.53.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1228241900,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1261781068,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3944230860,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4626828044,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.53.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1231050316,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.53.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 53,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1228241548,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.53.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 53,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3571323564,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.54.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1050193292,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.54.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5142316236,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.54.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1382572108,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.54.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3899338444,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.54.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1000050572,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.54.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2878240108,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.54.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1085117068,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.54.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4268450188,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5228734924,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5281839660,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2113744940,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.54.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	6144,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4268450284,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.54.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 54,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1081496972,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.54.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 54,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1057904012,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.55.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4214710668,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.55.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5029605836,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.55.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4191086988,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.55.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 935333772,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.55.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1261770828,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.55.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 965808012,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.55.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 55,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3740105068,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.55.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 959090572,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.55.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3130011468,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.55.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 890769292,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.55.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 55,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 808849292,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1568530508,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.56.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 759605132,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.56.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 629720972,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1492344908,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 582371212,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.56.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2741361580,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.56.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 787468172,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.56.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3742816620,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4838304716,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 741132172,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 761253772,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.56.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 742238092,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.56.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 56,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3406297004,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.56.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 56,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 725403532,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.57.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 557884460,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.57.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 557874220,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.57.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	3264
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3060543308,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.57.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2272332524,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.57.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 507739180,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.57.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 560516396,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.57.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 566552972,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.57.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4613613228,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1831537676,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 563153292,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1594263628,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.57.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5056824268,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.57.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 57,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 560516140,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.57.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 57,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4597884588,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.58.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2077395404,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.58.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 499979628,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.58.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 451646828,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 677070732,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 404297068,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.58.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 500973004,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.58.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 501103660,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.58.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 500972908,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 500973100,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3208170828,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 861769612,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.58.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4234043788,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.58.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 58,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2743327852,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.58.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 58,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1578534988,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.59.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1368279628,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.59.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 383161196,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.59.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 383150956,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.59.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 385783148,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.59.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 383140716,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.59.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3538209964,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.59.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 59,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1261770316,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.59.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 560516492,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.59.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 288441196,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.59.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 338576236,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.59.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 59,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2561823564,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1379756108,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.60.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4589408428,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.60.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 189015820,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1755352076,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.60.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3015807308,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.60.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 251275020,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.60.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4613628684,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.60.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3224083788,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 251290476,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	48,
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2877740908,
	"length_bytes": 491520
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 256656236,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.60.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 566642572,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.60.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 60,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5120197068,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.60.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 60,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 235546380,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.61.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 5029616588,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.61.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 147226028,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.61.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 58097068,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 102661548,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2883647020,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.61.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4094193100,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.61.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4094193196,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.61.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 163948204,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2079044140,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2364264780,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1686349484,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.61.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 164931340,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.61.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 61,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 163947948,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.61.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 61,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 148219308,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.62.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 100012,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.62.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 89772,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.62.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2697978923,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.62.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	17408,
	960
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2865925675,
	"length_bytes": 66846720
	},
	{
	"name": "language_model.model.layers.62.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2821361195,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.62.linear_attn.A_log",
	"role": "linear_attention_a_log",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3641872652,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.62.linear_attn.conv1d.weight",
	"role": "linear_attention_conv1d",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	10240,
	4,
	1
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 4803980236,
	"length_bytes": 81920
	},
	{
	"name": "language_model.model.layers.62.linear_attn.dt_bias",
	"role": "linear_attention_dt_bias",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	48
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 1752559020,
	"length_bytes": 96
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_a.weight",
	"role": "linear_attention_in_proj_a",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	48,
	960
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2076588,
	"length_bytes": 184320
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_b.weight",
	"role": "linear_attention_in_proj_b",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	48,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2268588,
	"length_bytes": 122880
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_qkv.weight",
	"role": "linear_attention_in_proj_qkv",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	10240,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 20741548,
	"length_bytes": 26214400
	},
	{
	"name": "language_model.model.layers.62.linear_attn.in_proj_z.weight",
	"role": "linear_attention_in_proj_z",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	6144,
	640
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 3374508,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.62.linear_attn.norm.weight",
	"role": "linear_attention_norm",
	"layer_index": 62,
	"dtype": "bf16",
	"shape": [
	128
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 2076332,
	"length_bytes": 256
	},
	{
	"name": "language_model.model.layers.62.linear_attn.out_proj.weight",
	"role": "linear_attention_out_proj",
	"layer_index": 62,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	768
	],
	"file": "model-00003-of-00004.safetensors",
	"offset_bytes": 793120652,
	"length_bytes": 15728640
	},
	{
	"name": "language_model.model.layers.63.self_attn.k_proj.weight",
	"role": "attention_k",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2769085995,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.63.self_attn.k_norm.weight",
	"role": "attention_k_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2690441771,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.63.input_layernorm.weight",
	"role": "attention_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2687646251,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.63.self_attn.o_proj.weight",
	"role": "attention_o",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 6
	},
	"shape": [
	5120,
	1152
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2742543403,
	"length_bytes": 23592960
	},
	{
	"name": "language_model.model.layers.63.post_attention_layernorm.weight",
	"role": "attention_post_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	5120
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2687636011,
	"length_bytes": 10240
	},
	{
	"name": "language_model.model.layers.63.self_attn.q_proj.weight",
	"role": "attention_q",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	12288,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2782193195,
	"length_bytes": 31457280
	},
	{
	"name": "language_model.model.layers.63.self_attn.q_norm.weight",
	"role": "attention_q_norm",
	"layer_index": 63,
	"dtype": "bf16",
	"shape": [
	256
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2769085483,
	"length_bytes": 512
	},
	{
	"name": "language_model.model.layers.63.self_attn.v_proj.weight",
	"role": "attention_v",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	1024,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2766300203,
	"length_bytes": 2621440
	},
	{
	"name": "language_model.model.layers.63.mlp.down_proj.weight",
	"role": "ffn_down",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	5120,
	2176
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2595721771,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.63.mlp.gate_proj.weight",
	"role": "ffn_gate",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2643071531,
	"length_bytes": 44564480
	},
	{
	"name": "language_model.model.layers.63.mlp.up_proj.weight",
	"role": "ffn_up",
	"layer_index": 63,
	"dtype": "u32",
	"source_quantized": true,
	"quantization": {
	"mode": "affine",
	"group_size": 64,
	"bits": 4
	},
	"shape": [
	17408,
	640
	],
	"file": "model-00004-of-00004.safetensors",
	"offset_bytes": 2548372011,
	"length_bytes": 44564480
	}
	]
	}