Add quality21 Ref2VA checkpoint and validation

NOTICE

@@ -2,12 +2,12 @@ MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyrig
 
 Modification notice
 
-The FL2VA safetensors checkpoint in this repository is a modified derivative of
-MiniMax H3. It was converted from the original Diffusers tensor layout to the
-ComfyUI layout, selectively quantized to INT8 ConvRot, and compressed with a
-4,097-row FP32 rank-16 table of the AdaLN time-conditioning curve. It uses the
-standard MiniMax-H3 table path in ComfyUI and does not require modified runtime
-code.
+The FL2VA and Ref2VA safetensors checkpoints in this repository are modified
+derivatives of MiniMax H3. They were converted from the original Diffusers
+tensor layout to the ComfyUI layout, selectively quantized to INT8 ConvRot, and
+compressed with separate 4,097-row FP32 rank-16 tables of their AdaLN
+time-conditioning curves. They use the standard MiniMax-H3 table path in
+ComfyUI and do not require modified runtime code.
 
 The two safetensors files under vae/ are unmodified, bit-identical copies of
 the Comfy-Org MiniMax-H3 ComfyUI repackages. They retain the MiniMax H3 license

README.md

@@ -14,42 +14,46 @@ tags:
   - video
   - audio
   - fl2va
+  - ref2va
 ---
 
 # MiniMax-H3 INT8 Lean ConvRot
 
 This repository contains a quality-oriented mixed-precision ComfyUI derivative
 of [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3).
-The currently released checkpoint is the **FL2VA** diffusion transformer. It is
-designed as a first 24 GiB GPU candidate while preserving more of the original
-model than the official pruned INT8 checkpoint.
+The released checkpoints cover both the **FL2VA** and **Ref2VA** diffusion
+transformers. Each is designed as a first 24 GiB GPU candidate while preserving
+more of the original model than the corresponding official pruned INT8
+checkpoint.
 
-The release is not an official MiniMax or ComfyOrg checkpoint. It uses the
-standard MiniMax-H3 `adaln_t_table` format and loads in an unmodified ComfyUI.
+These are not official MiniMax or ComfyOrg checkpoints. They use the standard
+MiniMax-H3 `adaln_t_table` format and load in an unmodified ComfyUI.
 
 ## Included files
 
 | Path | Purpose |
 |---|---|
 | [`FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors`](FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors) | Stock-compatible FL2VA diffusion transformer, 20.940 GiB |
+| [`Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors`](Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors) | Stock-compatible Ref2VA diffusion transformer, 20.940 GiB |
 | [`vae/minimax_h3_video_vae_fp16.safetensors`](vae/minimax_h3_video_vae_fp16.safetensors) | Stock ComfyUI MiniMax-H3 visual VAE, FP16, 4.850 GiB |
 | [`vae/minimax_h3_audio_vae_fp32.safetensors`](vae/minimax_h3_audio_vae_fp32.safetensors) | Stock ComfyUI MiniMax-H3 stereo audio VAE, FP32, 0.564 GiB |
-| [`reports/layer_policy.json`](reports/layer_policy.json) | Exact per-layer precision policy |
-| [`reports/validation.json`](reports/validation.json) | Structural, numerical, and CPU-load results |
+| [`reports/layer_policy.json`](reports/layer_policy.json) / [`layer_policy_ref2va.json`](reports/layer_policy_ref2va.json) | Exact per-layer precision policies |
+| [`reports/validation.json`](reports/validation.json) / [`validation_ref2va.json`](reports/validation_ref2va.json) | Structural, numerical, and CPU-load results |
 | [`reports/mm_quant_profile_fl2va.json`](reports/mm_quant_profile_fl2va.json) | Row-sampled reconstruction profile for all 200 main matrices |
+| [`reports/mm_quant_profile_ref2va.json`](reports/mm_quant_profile_ref2va.json) | Matching Ref2VA reconstruction profile |
 
 This repository intentionally does **not** include the MiniMax-H3 text encoder.
 It is being prepared as a separate repository. ComfyUI does not require the
 Diffusers scheduler, tokenizer, processor, or remote-code Python bundles as
-separate downloads: the standard workflow needs this diffusion model, the two
-included VAEs, and a compatible MiniMax-H3 text encoder. Ref2VA is not included
-in this release.
+separate downloads: a standard workflow needs one selected diffusion model,
+the two included VAEs, and a compatible MiniMax-H3 text encoder.
 
 ## What was changed
 
-The original FL2VA Diffusers shards contain 50 transformer blocks with separate
-Q, K, and V tensors. The converter streams those shards directly and performs
-the layout changes required by current ComfyUI MiniMax-H3 support:
+The original FL2VA and Ref2VA Diffusers shards each contain 50 transformer
+blocks with separate Q, K, and V tensors. The converter streams the selected
+variant directly and performs the layout changes required by current ComfyUI
+MiniMax-H3 support:
 
 - packs attention rows as global `cat(Q, K, V)`;
 - swaps the two `fc1` halves from `[value, gate]` to `[gate, value]`;
@@ -65,14 +69,14 @@ the layout changes required by current ComfyUI MiniMax-H3 support:
 - replaces only the redundant input width of the 51 large AdaLN projections,
   while keeping every block and final AdaLN projection separate in FP32.
 
-The checkpoint contains 872 tensor entries: 170 INT8 weights, 170 quantization
-configuration blobs, 250 BF16 entries, and 282 FP32 entries.
+Each checkpoint contains 872 tensor entries: 170 INT8 weights, 170
+quantization configuration blobs, 250 BF16 entries, and 282 FP32 entries.
 
 ## Precision policy
 
 Every DiT block has four main matrices: QKV, attention output, MLP `fc1`, and
-MLP `fc2`, for 200 matrices total. The `quality21` profile stores 170 in INT8
-and 30 in BF16.
+MLP `fc2`, for 200 matrices total per checkpoint. The `quality21` profile stores
+170 in INT8 and 30 in BF16.
 
 The BF16 island is:
 
@@ -80,11 +84,13 @@ The BF16 island is:
   and 49 (27 matrices);
 - MLP `fc2` in blocks 39, 45, and 49 (3 matrices).
 
-These were selected from a deterministic reconstruction audit using 64 output
-rows and every input column of each matrix. Attention output projections were
-the most error-prone family and are considerably cheaper to retain in BF16 than
-QKV or `fc1`. The exact machine-readable list is in
-[`reports/layer_policy.json`](reports/layer_policy.json).
+These were selected from deterministic reconstruction audits using 64 output
+rows and every input column of each matrix. Ref2VA independently selected the
+same 30-layer set as FL2VA, with a per-layer error correlation of `0.999676`.
+Attention output projections were the most error-prone family and are
+considerably cheaper to retain in BF16 than QKV or `fc1`. The exact
+machine-readable lists are in [`reports/layer_policy.json`](reports/layer_policy.json)
+and [`reports/layer_policy_ref2va.json`](reports/layer_policy_ref2va.json).
 
 ## Time conditioning and the AdaLN table
 
@@ -108,10 +114,11 @@ Each of the 50 block AdaLN projections and the final AdaLN projection remains a
 different FP32 linear layer. Only their shared input curve is compressed.
 
 On 19 validation timesteps, the relative AdaLN output error against the raw HF
-weights was `3.04e-7` to `3.82e-7` for blocks 0, 24, 49, and the final layer.
-For context, the official pruned rank-8 FP16 table measured about
-`0.016%–0.021%` on the same projections. This comparison covers only the
-time/AdaLN path, not end-to-end generation quality.
+weights was `3.04e-7` to `3.82e-7` for FL2VA and `3.42e-7` to `4.46e-7` for
+Ref2VA across blocks 0, 24, 49, and the final layer. For context, the official
+pruned rank-8 FP16 tables measured about `0.016%–0.021%` on the same
+projections. This comparison covers only the time/AdaLN path, not end-to-end
+generation quality.
 
 ## Comparison with official ComfyOrg files
 
@@ -119,38 +126,44 @@ time/AdaLN path, not end-to-end generation quality.
 |---|---:|---:|---|
 | Official regular INT8 ConvRot | about 31.70 GiB | 200 INT8 | full AdaLN weights |
 | Official pruned INT8 ConvRot | 19.530 GiB | 200 INT8 | FP16 rank-8 table |
-| This `quality21` release | 20.940 GiB | 170 INT8 + 30 BF16 | FP32 rank-16 table, 4,097 rows |
+| Each `quality21` release | 20.940 GiB | 170 INT8 + 30 BF16 | FP32 rank-16 table, 4,097 rows |
 
-This file is 1.410 GiB (7.22%) larger than the official pruned INT8 checkpoint.
-The added space is intentionally spent on a much denser, higher-precision time
-curve and a BF16 island for the most sensitive main matrices.
+Each file is about 1.410 GiB (7.22%) larger than its official pruned INT8
+counterpart. The added space is intentionally spent on a much denser,
+higher-precision time curve and a BF16 island for the most sensitive main
+matrices.
 
-The full 200-matrix sampled reconstruction proxy produced:
+The full 200-matrix sampled reconstruction proxies produced:
 
-| Metric | Official-style absmax INT8 | This profile |
+| Variant | Official-style absmax INT8 | This profile |
 |---|---:|---:|
-| Unweighted mean relative matrix error | 0.9345% | 0.7639% |
+| FL2VA | 0.9345% | 0.7639% |
+| Ref2VA | 0.9345% | 0.7639% |
 
-For this profile, selected BF16 matrices count as zero storage reconstruction
+For these profiles, selected BF16 matrices count as zero storage reconstruction
 error. On the remaining 170 INT8 matrices, searched scales reduced the mean
-proxy from 0.9109% to 0.8987%. The overall 18.26% reduction is a sampled weight
-reconstruction result, **not** a perceptual or generation benchmark.
+proxy from about 0.9109% to 0.8987%. The approximately 18.25% overall reduction
+is a sampled weight reconstruction result, **not** a perceptual or generation
+benchmark.
 
 ## ComfyUI installation
 
 1. Use a current ComfyUI checkout with MiniMax-H3 and native INT8 ConvRot
    support. This release was CPU-load-tested on a clean, unmodified ComfyUI
    commit `14b05228cef127ce529bc0c08660770d4af3e9a8`.
-2. Place the FL2VA safetensors file in
-   `ComfyUI/models/diffusion_models/`.
+2. Place the FL2VA and/or Ref2VA safetensors file in
+   `ComfyUI/models/diffusion_models/`. FL2VA is for text/image first-last-frame
+   workflows; Ref2VA is for reference image/video/audio workflows.
 3. Place both files from `vae/` in `ComfyUI/models/vae/`.
-4. Load the three files with the standard diffusion-model and VAE loaders in a
-   MiniMax-H3 FL2VA workflow. Supply a compatible text encoder separately.
+4. Load one diffusion checkpoint and both VAEs with the standard loaders in the
+   matching MiniMax-H3 FL2VA or Ref2VA workflow. Supply a compatible text
+   encoder separately.
 
 ```text
 ComfyUI/models/
 |-- diffusion_models/
-|   `-- minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors
+|   |-- minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors
+|   `-- minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors
 |-- vae/
 |   |-- minimax_h3_video_vae_fp16.safetensors
 |   `-- minimax_h3_audio_vae_fp32.safetensors
@@ -158,21 +171,28 @@ ComfyUI/models/
     `-- <MiniMax-H3 text encoder from its separate repository>
 ```
 
-No custom node or core patch is required for this checkpoint.
+No custom node or core patch is required for either checkpoint.
 
 ## Validation status
 
 Completed:
 
 - safetensors structure, shapes, dtypes, and all 170 ConvRot descriptors;
-- SHA-256: `ad0cf8b50ba92068b5a4477527d5da73412a22db99cded35b07195504e78cf6a`;
+- SHA-256: FL2VA
+  `ad0cf8b50ba92068b5a4477527d5da73412a22db99cded35b07195504e78cf6a`;
+  Ref2VA
+  `f4b2a8b4a53446f5d696a39deb693b09ac7ce37013793a8b19340644e38b154f`;
 - BF16 main island: 30/30 matrices bitwise equal to raw HF;
-- 871/871 tensors outside the time representation preserved bit-for-bit during
-  the final stock-table conversion;
-- FP32 table grid reproduced exactly and interpolation checked at every one of
-  the 4,096 interval midpoints;
+- FL2VA: 871/871 tensors outside the time representation preserved bit-for-bit
+  during the final stock-table conversion;
+- Ref2VA: 258/258 direct source-copy tensors (4,146,503,680 bytes) bitwise equal
+  to raw HF;
+- FP32 table interpolation checked at every one of the 4,096 interval
+  midpoints; FL2VA grid rows reproduced exactly, and the independent Ref2VA
+  FP64-SVD rebuild differed by at most `4.55e-13`;
 - full CPU load through a clean, unmodified ComfyUI as
-  `ModelPatcher -> MiniMaxH3Model`, table mode `(4097, 16)`.
+  `ModelPatcher -> MiniMaxH3Model`, table mode `(4097, 16)`, for both diffusion
+  checkpoints;
 - both companion VAE files fully loaded on CPU through the same clean ComfyUI:
   `MiniMaxH3VideoVAE` in FP16 with native internal tiling, and
   `MiniMaxH3AudioVAE` in FP32 with 32 kHz stereo output;
@@ -184,9 +204,9 @@ Completed:
 
 Not completed at release time:
 
-- CUDA/GPU loading of this exact checkpoint;
+- CUDA/GPU loading of these exact checkpoints;
 - peak VRAM measurement for a target resolution and frame count;
-- end-to-end FL2VA generation;
+- end-to-end FL2VA or Ref2VA generation;
 - same-seed visual/audio A/B against BF16 and official pruned variants.
 
 Therefore, 20.940 GiB is the on-disk tensor payload, not a promise that every
@@ -195,15 +215,18 @@ frame count, batch size, and offloading policy determine peak VRAM.
 
 ## Conversion provenance
 
-The converter reads the original top-level `transformer/` Diffusers shards from
-MiniMaxAI directly; it does not create a 61+ GiB merged BF16 intermediate. The
-nested task-specific transformer folder was not used because its already-packed
-QKV is head-major, while current ComfyUI expects global `cat(Q,K,V)` packing.
+The converter reads the original top-level `transformer/` or `transformer_ref/`
+Diffusers shards from MiniMaxAI directly; it does not create a 61+ GiB merged
+BF16 intermediate. The nested task-specific transformer folders were not used
+because their already-packed QKV is head-major, while current ComfyUI expects
+global `cat(Q,K,V)` packing.
 
-The official
+The official FL2VA and Ref2VA pruned files,
 [`minimax_h3_fl2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors)
-was used only as a layout reference and as the source of the FP32
-`rope.inv_freq` tensor absent from the raw Diffusers state dict.
+and
+[`minimax_h3_ref2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors),
+were used only as layout references and as the sources of the corresponding
+FP32 `rope.inv_freq` tensors absent from the raw Diffusers state dicts.
 
 The two files under `vae/` are unmodified, bit-identical copies of the standard
 [`Comfy-Org/MiniMax-H3`](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/vae)
@@ -220,8 +243,9 @@ latent statistics. No remote Python code or external VAE config is required.
 
 This release is intended for research and local ComfyUI inference. It is a
 new, structurally validated operating point, not a completed quality benchmark.
-The current layer policy was profiled on FL2VA and must not be assumed optimal
-for Ref2VA without a separate audit.
+Both variants were profiled separately. Their independently selected
+quality21 BF16 islands are identical, but this does not establish perceptual
+quality without end-to-end A/B generation.
 
 ## License
 

reports/layer_policy_ref2va.json

@@ -0,0 +1,57 @@
+{
+  "variant": "Ref2VA",
+  "profile": "quality21",
+  "main_matrix_count": 200,
+  "int8_convrot_group_256_count": 170,
+  "bf16_main_count": 30,
+  "bf16_main_layers": [
+    "blocks.0.attn.out_proj",
+    "blocks.1.attn.out_proj",
+    "blocks.2.attn.out_proj",
+    "blocks.3.attn.out_proj",
+    "blocks.4.attn.out_proj",
+    "blocks.5.attn.out_proj",
+    "blocks.6.attn.out_proj",
+    "blocks.7.attn.out_proj",
+    "blocks.8.attn.out_proj",
+    "blocks.9.attn.out_proj",
+    "blocks.10.attn.out_proj",
+    "blocks.11.attn.out_proj",
+    "blocks.12.attn.out_proj",
+    "blocks.13.attn.out_proj",
+    "blocks.14.attn.out_proj",
+    "blocks.15.attn.out_proj",
+    "blocks.17.attn.out_proj",
+    "blocks.19.attn.out_proj",
+    "blocks.20.attn.out_proj",
+    "blocks.27.attn.out_proj",
+    "blocks.38.attn.out_proj",
+    "blocks.43.attn.out_proj",
+    "blocks.44.attn.out_proj",
+    "blocks.45.attn.out_proj",
+    "blocks.46.attn.out_proj",
+    "blocks.47.attn.out_proj",
+    "blocks.49.attn.out_proj",
+    "blocks.39.mlp.fc2",
+    "blocks.45.mlp.fc2",
+    "blocks.49.mlp.fc2"
+  ],
+  "selection_validation": {
+    "method": "deterministic 64 output rows per matrix, all input columns",
+    "fl2va_ref2va_error_correlation": 0.9996760487556458,
+    "same_layers_as_fl2va": true
+  },
+  "other_precision": {
+    "token_refiner": "source BF16",
+    "normalization_and_qk_norm": "source BF16",
+    "condition_projection": "source BF16",
+    "video_audio_patch_and_output_projections": "source FP32",
+    "adaln_projections": "51 separate FP32 rank-16 projections",
+    "adaln_time_table": "FP32 [4097, 16], generated from the original Ref2VA FP32 time embedder and shared rank-16 basis",
+    "rope_inv_freq": "FP32 [16]"
+  },
+  "packing": {
+    "qkv": "global cat(Q,K,V) for current ComfyUI MiniMax-H3 split",
+    "mlp_fc1": "halves reordered from [value,gate] to [gate,value]"
+  }
+}

reports/mm_quant_profile_ref2va.json

@@ -0,0 +1,3011 @@
+{
+  "variant": "ref2va",
+  "source": "F:\\AI\\hf_originals\\MiniMax__MiniMax-H3\\HF\\transformer_ref",
+  "groupsize": 256,
+  "rows": 64,
+  "search_min": 0.8,
+  "clip_margin": 0.05,
+  "elapsed_seconds": 66.3423388004303,
+  "layers": [
+    {
+      "layer": "blocks.0.attn.qkv_proj",
+      "block": 0,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009149877354502678,
+      "searched_error": 0.009098502807319164,
+      "improvement": 5.137454718351364e-05,
+      "clipped_row_fraction": 0.109375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.0.attn.out_proj",
+      "block": 0,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.011861145496368408,
+      "searched_error": 0.01176026277244091,
+      "improvement": 0.00010088272392749786,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.0.mlp.fc1",
+      "block": 0,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008755319751799107,
+      "searched_error": 0.00868719257414341,
+      "improvement": 6.812717765569687e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.0.mlp.fc2",
+      "block": 0,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009267843328416348,
+      "searched_error": 0.00897966418415308,
+      "improvement": 0.0002881791442632675,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.1.attn.qkv_proj",
+      "block": 1,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008966601453721523,
+      "searched_error": 0.008890115655958652,
+      "improvement": 7.648579776287079e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.1.attn.out_proj",
+      "block": 1,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.012773617170751095,
+      "searched_error": 0.012632046826183796,
+      "improvement": 0.0001415703445672989,
+      "clipped_row_fraction": 0.390625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.1.mlp.fc1",
+      "block": 1,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00876633170992136,
+      "searched_error": 0.008713556453585625,
+      "improvement": 5.277525633573532e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.1.mlp.fc2",
+      "block": 1,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009261968545615673,
+      "searched_error": 0.008988389745354652,
+      "improvement": 0.00027357880026102066,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.2.attn.qkv_proj",
+      "block": 2,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008780846372246742,
+      "searched_error": 0.00874966662377119,
+      "improvement": 3.1179748475551605e-05,
+      "clipped_row_fraction": 0.109375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.2.attn.out_proj",
+      "block": 2,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.012882267124950886,
+      "searched_error": 0.01274180319160223,
+      "improvement": 0.0001404639333486557,
+      "clipped_row_fraction": 0.328125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.2.mlp.fc1",
+      "block": 2,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008909977041184902,
+      "searched_error": 0.008883747272193432,
+      "improvement": 2.6229768991470337e-05,
+      "clipped_row_fraction": 0.109375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.2.mlp.fc2",
+      "block": 2,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009286819025874138,
+      "searched_error": 0.009042869322001934,
+      "improvement": 0.00024394970387220383,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.3.attn.qkv_proj",
+      "block": 3,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008728427812457085,
+      "searched_error": 0.008676868863403797,
+      "improvement": 5.1558949053287506e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.3.attn.out_proj",
+      "block": 3,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.011711601167917252,
+      "searched_error": 0.011610935442149639,
+      "improvement": 0.00010066572576761246,
+      "clipped_row_fraction": 0.359375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.3.mlp.fc1",
+      "block": 3,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008689207956194878,
+      "searched_error": 0.008630858734250069,
+      "improvement": 5.834922194480896e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.3.mlp.fc2",
+      "block": 3,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009451334364712238,
+      "searched_error": 0.00917091965675354,
+      "improvement": 0.00028041470795869827,
+      "clipped_row_fraction": 0.796875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.4.attn.qkv_proj",
+      "block": 4,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008999093435704708,
+      "searched_error": 0.008943070657551289,
+      "improvement": 5.6022778153419495e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.4.attn.out_proj",
+      "block": 4,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010633870959281921,
+      "searched_error": 0.01052608247846365,
+      "improvement": 0.00010778848081827164,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.4.mlp.fc1",
+      "block": 4,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008890100754797459,
+      "searched_error": 0.00882396474480629,
+      "improvement": 6.613600999116898e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.4.mlp.fc2",
+      "block": 4,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009374679997563362,
+      "searched_error": 0.00907217524945736,
+      "improvement": 0.0003025047481060028,
+      "clipped_row_fraction": 0.875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.5.attn.qkv_proj",
+      "block": 5,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008881070651113987,
+      "searched_error": 0.00881462637335062,
+      "improvement": 6.64442777633667e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.5.attn.out_proj",
+      "block": 5,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.011126119643449783,
+      "searched_error": 0.011061684228479862,
+      "improvement": 6.443541496992111e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.5.mlp.fc1",
+      "block": 5,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008905366994440556,
+      "searched_error": 0.00883188284933567,
+      "improvement": 7.34841451048851e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.5.mlp.fc2",
+      "block": 5,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009375902824103832,
+      "searched_error": 0.009108574129641056,
+      "improvement": 0.0002673286944627762,
+      "clipped_row_fraction": 0.75,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.6.attn.qkv_proj",
+      "block": 6,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008814658969640732,
+      "searched_error": 0.008784820325672626,
+      "improvement": 2.9838643968105316e-05,
+      "clipped_row_fraction": 0.125,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.6.attn.out_proj",
+      "block": 6,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010695223696529865,
+      "searched_error": 0.010601626709103584,
+      "improvement": 9.359698742628098e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.6.mlp.fc1",
+      "block": 6,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008920718915760517,
+      "searched_error": 0.008867485448718071,
+      "improvement": 5.3233467042446136e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.6.mlp.fc2",
+      "block": 6,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009407012723386288,
+      "searched_error": 0.00913174543529749,
+      "improvement": 0.0002752672880887985,
+      "clipped_row_fraction": 0.8125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.7.attn.qkv_proj",
+      "block": 7,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00881594605743885,
+      "searched_error": 0.008770952932536602,
+      "improvement": 4.499312490224838e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.7.attn.out_proj",
+      "block": 7,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.011022337712347507,
+      "searched_error": 0.010928143747150898,
+      "improvement": 9.41939651966095e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.7.mlp.fc1",
+      "block": 7,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008879266679286957,
+      "searched_error": 0.008849864825606346,
+      "improvement": 2.9401853680610657e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.7.mlp.fc2",
+      "block": 7,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009405774064362049,
+      "searched_error": 0.009109561331570148,
+      "improvement": 0.00029621273279190063,
+      "clipped_row_fraction": 0.8125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.8.attn.qkv_proj",
+      "block": 8,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009008223190903664,
+      "searched_error": 0.0089647201821208,
+      "improvement": 4.350300878286362e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.8.attn.out_proj",
+      "block": 8,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.01042734831571579,
+      "searched_error": 0.010339769534766674,
+      "improvement": 8.757878094911575e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.8.mlp.fc1",
+      "block": 8,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008798491209745407,
+      "searched_error": 0.008746922016143799,
+      "improvement": 5.1569193601608276e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.8.mlp.fc2",
+      "block": 8,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009286905638873577,
+      "searched_error": 0.009002236649394035,
+      "improvement": 0.0002846689894795418,
+      "clipped_row_fraction": 0.84375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.9.attn.qkv_proj",
+      "block": 9,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008678901940584183,
+      "searched_error": 0.008630734868347645,
+      "improvement": 4.816707223653793e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.9.attn.out_proj",
+      "block": 9,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010669376701116562,
+      "searched_error": 0.010587197728455067,
+      "improvement": 8.217897266149521e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.9.mlp.fc1",
+      "block": 9,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008611093275249004,
+      "searched_error": 0.008564319461584091,
+      "improvement": 4.677381366491318e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.9.mlp.fc2",
+      "block": 9,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009352547116577625,
+      "searched_error": 0.009089149534702301,
+      "improvement": 0.00026339758187532425,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.10.attn.qkv_proj",
+      "block": 10,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00882525835186243,
+      "searched_error": 0.008772222325205803,
+      "improvement": 5.3036026656627655e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.10.attn.out_proj",
+      "block": 10,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.01042264699935913,
+      "searched_error": 0.010306568816304207,
+      "improvement": 0.00011607818305492401,
+      "clipped_row_fraction": 0.375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.10.mlp.fc1",
+      "block": 10,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008798566646873951,
+      "searched_error": 0.008739395067095757,
+      "improvement": 5.917157977819443e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.10.mlp.fc2",
+      "block": 10,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009307919070124626,
+      "searched_error": 0.00904966052621603,
+      "improvement": 0.00025825854390859604,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.11.attn.qkv_proj",
+      "block": 11,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.0088732298463583,
+      "searched_error": 0.008829551748931408,
+      "improvement": 4.367809742689133e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.11.attn.out_proj",
+      "block": 11,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010403656400740147,
+      "searched_error": 0.01028230506926775,
+      "improvement": 0.00012135133147239685,
+      "clipped_row_fraction": 0.359375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.11.mlp.fc1",
+      "block": 11,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008903699927031994,
+      "searched_error": 0.008855773136019707,
+      "improvement": 4.792679101228714e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.11.mlp.fc2",
+      "block": 11,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009393034502863884,
+      "searched_error": 0.009126064367592335,
+      "improvement": 0.0002669701352715492,
+      "clipped_row_fraction": 0.734375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.12.attn.qkv_proj",
+      "block": 12,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008730018511414528,
+      "searched_error": 0.008657277561724186,
+      "improvement": 7.274094969034195e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.12.attn.out_proj",
+      "block": 12,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010259442962706089,
+      "searched_error": 0.010190111584961414,
+      "improvement": 6.933137774467468e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.12.mlp.fc1",
+      "block": 12,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008771385997533798,
+      "searched_error": 0.00874131079763174,
+      "improvement": 3.0075199902057648e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.12.mlp.fc2",
+      "block": 12,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009368893690407276,
+      "searched_error": 0.009100206196308136,
+      "improvement": 0.00026868749409914017,
+      "clipped_row_fraction": 0.75,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.13.attn.qkv_proj",
+      "block": 13,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008816634304821491,
+      "searched_error": 0.008743951097130775,
+      "improvement": 7.268320769071579e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.13.attn.out_proj",
+      "block": 13,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010227742604911327,
+      "searched_error": 0.010140802711248398,
+      "improvement": 8.693989366292953e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.13.mlp.fc1",
+      "block": 13,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00874239206314087,
+      "searched_error": 0.008691736496984959,
+      "improvement": 5.065556615591049e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.13.mlp.fc2",
+      "block": 13,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009334002621471882,
+      "searched_error": 0.009064814075827599,
+      "improvement": 0.0002691885456442833,
+      "clipped_row_fraction": 0.796875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.14.attn.qkv_proj",
+      "block": 14,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008730604313313961,
+      "searched_error": 0.008698868565261364,
+      "improvement": 3.1735748052597046e-05,
+      "clipped_row_fraction": 0.109375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.14.attn.out_proj",
+      "block": 14,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010583301074802876,
+      "searched_error": 0.010479689575731754,
+      "improvement": 0.00010361149907112122,
+      "clipped_row_fraction": 0.359375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.14.mlp.fc1",
+      "block": 14,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00874729547649622,
+      "searched_error": 0.008711329661309719,
+      "improvement": 3.596581518650055e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.14.mlp.fc2",
+      "block": 14,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009479190222918987,
+      "searched_error": 0.009177664294838905,
+      "improvement": 0.00030152592808008194,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.15.attn.qkv_proj",
+      "block": 15,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008874666877090931,
+      "searched_error": 0.008819757029414177,
+      "improvement": 5.4909847676754e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.15.attn.out_proj",
+      "block": 15,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.011003787629306316,
+      "searched_error": 0.010917904786765575,
+      "improvement": 8.588284254074097e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.15.mlp.fc1",
+      "block": 15,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008837355300784111,
+      "searched_error": 0.00877710897475481,
+      "improvement": 6.024632602930069e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.15.mlp.fc2",
+      "block": 15,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009284839034080505,
+      "searched_error": 0.008988245390355587,
+      "improvement": 0.00029659364372491837,
+      "clipped_row_fraction": 0.84375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.16.attn.qkv_proj",
+      "block": 16,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008748017251491547,
+      "searched_error": 0.008695828728377819,
+      "improvement": 5.218852311372757e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.16.attn.out_proj",
+      "block": 16,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00988137535750866,
+      "searched_error": 0.009816288948059082,
+      "improvement": 6.508640944957733e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.16.mlp.fc1",
+      "block": 16,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008817206136882305,
+      "searched_error": 0.008756496012210846,
+      "improvement": 6.07101246714592e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.16.mlp.fc2",
+      "block": 16,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009356603026390076,
+      "searched_error": 0.009065966121852398,
+      "improvement": 0.00029063690453767776,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.17.attn.qkv_proj",
+      "block": 17,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008703425526618958,
+      "searched_error": 0.008664635941386223,
+      "improvement": 3.878958523273468e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.17.attn.out_proj",
+      "block": 17,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010421531274914742,
+      "searched_error": 0.01031559333205223,
+      "improvement": 0.00010593794286251068,
+      "clipped_row_fraction": 0.390625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.17.mlp.fc1",
+      "block": 17,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00872599147260189,
+      "searched_error": 0.00865576695650816,
+      "improvement": 7.022451609373093e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.17.mlp.fc2",
+      "block": 17,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009331951849162579,
+      "searched_error": 0.009054707363247871,
+      "improvement": 0.0002772444859147072,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.18.attn.qkv_proj",
+      "block": 18,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008934194222092628,
+      "searched_error": 0.008866879157721996,
+      "improvement": 6.731506437063217e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.18.attn.out_proj",
+      "block": 18,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009590333327651024,
+      "searched_error": 0.009500373154878616,
+      "improvement": 8.996017277240753e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.18.mlp.fc1",
+      "block": 18,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.0087647195905447,
+      "searched_error": 0.00870244950056076,
+      "improvement": 6.227008998394012e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.18.mlp.fc2",
+      "block": 18,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009473690763115883,
+      "searched_error": 0.009212583303451538,
+      "improvement": 0.0002611074596643448,
+      "clipped_row_fraction": 0.75,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.19.attn.qkv_proj",
+      "block": 19,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00883630383759737,
+      "searched_error": 0.008758751675486565,
+      "improvement": 7.755216211080551e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.19.attn.out_proj",
+      "block": 19,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010613351128995419,
+      "searched_error": 0.010535142384469509,
+      "improvement": 7.820874452590942e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.19.mlp.fc1",
+      "block": 19,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008808483369648457,
+      "searched_error": 0.00879175029695034,
+      "improvement": 1.6733072698116302e-05,
+      "clipped_row_fraction": 0.09375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.19.mlp.fc2",
+      "block": 19,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009467557072639465,
+      "searched_error": 0.009168003685772419,
+      "improvement": 0.00029955338686704636,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.20.attn.qkv_proj",
+      "block": 20,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008711456321179867,
+      "searched_error": 0.00862672459334135,
+      "improvement": 8.473172783851624e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.20.attn.out_proj",
+      "block": 20,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010097628459334373,
+      "searched_error": 0.010016809217631817,
+      "improvement": 8.081924170255661e-05,
+      "clipped_row_fraction": 0.328125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.20.mlp.fc1",
+      "block": 20,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008967501111328602,
+      "searched_error": 0.008894388563930988,
+      "improvement": 7.311254739761353e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.20.mlp.fc2",
+      "block": 20,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009428652003407478,
+      "searched_error": 0.00914465170353651,
+      "improvement": 0.00028400029987096786,
+      "clipped_row_fraction": 0.796875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.21.attn.qkv_proj",
+      "block": 21,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.0088729839771986,
+      "searched_error": 0.00883404165506363,
+      "improvement": 3.894232213497162e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.21.attn.out_proj",
+      "block": 21,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009565706364810467,
+      "searched_error": 0.009480264037847519,
+      "improvement": 8.544232696294785e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.21.mlp.fc1",
+      "block": 21,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00895257480442524,
+      "searched_error": 0.00890747457742691,
+      "improvement": 4.510022699832916e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.21.mlp.fc2",
+      "block": 21,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009270595386624336,
+      "searched_error": 0.008980251848697662,
+      "improvement": 0.0002903435379266739,
+      "clipped_row_fraction": 0.84375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.22.attn.qkv_proj",
+      "block": 22,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00894355308264494,
+      "searched_error": 0.00891187135130167,
+      "improvement": 3.168173134326935e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.22.attn.out_proj",
+      "block": 22,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00975025724619627,
+      "searched_error": 0.009666379541158676,
+      "improvement": 8.387770503759384e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.22.mlp.fc1",
+      "block": 22,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00883104931563139,
+      "searched_error": 0.008784261532127857,
+      "improvement": 4.678778350353241e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.22.mlp.fc2",
+      "block": 22,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009395661763846874,
+      "searched_error": 0.009091234765946865,
+      "improvement": 0.00030442699790000916,
+      "clipped_row_fraction": 0.875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.23.attn.qkv_proj",
+      "block": 23,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00885548535734415,
+      "searched_error": 0.008821829222142696,
+      "improvement": 3.365613520145416e-05,
+      "clipped_row_fraction": 0.125,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.23.attn.out_proj",
+      "block": 23,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009813772514462471,
+      "searched_error": 0.009709117002785206,
+      "improvement": 0.00010465551167726517,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.23.mlp.fc1",
+      "block": 23,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008615493774414062,
+      "searched_error": 0.008556989952921867,
+      "improvement": 5.850382149219513e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.23.mlp.fc2",
+      "block": 23,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00936894677579403,
+      "searched_error": 0.009070239961147308,
+      "improvement": 0.0002987068146467209,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.24.attn.qkv_proj",
+      "block": 24,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008828956633806229,
+      "searched_error": 0.008763122372329235,
+      "improvement": 6.583426147699356e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.24.attn.out_proj",
+      "block": 24,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00980150606483221,
+      "searched_error": 0.009715952910482883,
+      "improvement": 8.555315434932709e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.24.mlp.fc1",
+      "block": 24,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008843949995934963,
+      "searched_error": 0.008792220614850521,
+      "improvement": 5.172938108444214e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.24.mlp.fc2",
+      "block": 24,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009455005638301373,
+      "searched_error": 0.009169526398181915,
+      "improvement": 0.00028547924011945724,
+      "clipped_row_fraction": 0.8125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.25.attn.qkv_proj",
+      "block": 25,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00900735892355442,
+      "searched_error": 0.008946533314883709,
+      "improvement": 6.082560867071152e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.25.attn.out_proj",
+      "block": 25,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009812835603952408,
+      "searched_error": 0.00972868874669075,
+      "improvement": 8.414685726165771e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.25.mlp.fc1",
+      "block": 25,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008857025764882565,
+      "searched_error": 0.008804724551737309,
+      "improvement": 5.230121314525604e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.25.mlp.fc2",
+      "block": 25,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009359694086015224,
+      "searched_error": 0.009061169810593128,
+      "improvement": 0.00029852427542209625,
+      "clipped_row_fraction": 0.875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.26.attn.qkv_proj",
+      "block": 26,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008898529224097729,
+      "searched_error": 0.008845372125506401,
+      "improvement": 5.315709859132767e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.26.attn.out_proj",
+      "block": 26,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010012996383011341,
+      "searched_error": 0.009899193421006203,
+      "improvement": 0.0001138029620051384,
+      "clipped_row_fraction": 0.359375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.26.mlp.fc1",
+      "block": 26,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008813909254968166,
+      "searched_error": 0.008754000067710876,
+      "improvement": 5.9909187257289886e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.26.mlp.fc2",
+      "block": 26,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009220638312399387,
+      "searched_error": 0.008964918553829193,
+      "improvement": 0.00025571975857019424,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.27.attn.qkv_proj",
+      "block": 27,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008921593427658081,
+      "searched_error": 0.008879701606929302,
+      "improvement": 4.189182072877884e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.27.attn.out_proj",
+      "block": 27,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010251954197883606,
+      "searched_error": 0.01017802581191063,
+      "improvement": 7.392838597297668e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.27.mlp.fc1",
+      "block": 27,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008891111239790916,
+      "searched_error": 0.008837737143039703,
+      "improvement": 5.3374096751213074e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.27.mlp.fc2",
+      "block": 27,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009307934902608395,
+      "searched_error": 0.009034196846187115,
+      "improvement": 0.0002737380564212799,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.28.attn.qkv_proj",
+      "block": 28,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008664083667099476,
+      "searched_error": 0.008631406351923943,
+      "improvement": 3.2677315175533295e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.28.attn.out_proj",
+      "block": 28,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010012027807533741,
+      "searched_error": 0.009910239838063717,
+      "improvement": 0.00010178796947002411,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.28.mlp.fc1",
+      "block": 28,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00868856068700552,
+      "searched_error": 0.008628825657069683,
+      "improvement": 5.973502993583679e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.28.mlp.fc2",
+      "block": 28,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009298889897763729,
+      "searched_error": 0.00900914054363966,
+      "improvement": 0.0002897493541240692,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.29.attn.qkv_proj",
+      "block": 29,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008712724782526493,
+      "searched_error": 0.008658976294100285,
+      "improvement": 5.3748488426208496e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.29.attn.out_proj",
+      "block": 29,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009863519109785557,
+      "searched_error": 0.009750662371516228,
+      "improvement": 0.00011285673826932907,
+      "clipped_row_fraction": 0.375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.29.mlp.fc1",
+      "block": 29,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008687011897563934,
+      "searched_error": 0.00863416213542223,
+      "improvement": 5.284976214170456e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.29.mlp.fc2",
+      "block": 29,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00952060241252184,
+      "searched_error": 0.009227273054420948,
+      "improvement": 0.0002933293581008911,
+      "clipped_row_fraction": 0.84375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.30.attn.qkv_proj",
+      "block": 30,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008769126608967781,
+      "searched_error": 0.008718494325876236,
+      "improvement": 5.0632283091545105e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.30.attn.out_proj",
+      "block": 30,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00980446208268404,
+      "searched_error": 0.009715707041323185,
+      "improvement": 8.87550413608551e-05,
+      "clipped_row_fraction": 0.328125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.30.mlp.fc1",
+      "block": 30,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008770755492150784,
+      "searched_error": 0.008700021542608738,
+      "improvement": 7.07339495420456e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.30.mlp.fc2",
+      "block": 30,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00926683098077774,
+      "searched_error": 0.008997678756713867,
+      "improvement": 0.0002691522240638733,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.31.attn.qkv_proj",
+      "block": 31,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008715620264410973,
+      "searched_error": 0.008677283301949501,
+      "improvement": 3.833696246147156e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.31.attn.out_proj",
+      "block": 31,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010006719268858433,
+      "searched_error": 0.009893168695271015,
+      "improvement": 0.0001135505735874176,
+      "clipped_row_fraction": 0.375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.31.mlp.fc1",
+      "block": 31,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008751807734370232,
+      "searched_error": 0.008717701770365238,
+      "improvement": 3.410596400499344e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.31.mlp.fc2",
+      "block": 31,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009457345120608807,
+      "searched_error": 0.009170906618237495,
+      "improvement": 0.0002864385023713112,
+      "clipped_row_fraction": 0.8125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.32.attn.qkv_proj",
+      "block": 32,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008660205639898777,
+      "searched_error": 0.008621814660727978,
+      "improvement": 3.8390979170799255e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.32.attn.out_proj",
+      "block": 32,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009854556061327457,
+      "searched_error": 0.009778095409274101,
+      "improvement": 7.646065205335617e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.32.mlp.fc1",
+      "block": 32,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008708640933036804,
+      "searched_error": 0.00866017211228609,
+      "improvement": 4.846882075071335e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.32.mlp.fc2",
+      "block": 32,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009446533396840096,
+      "searched_error": 0.009172597900032997,
+      "improvement": 0.0002739354968070984,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.33.attn.qkv_proj",
+      "block": 33,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008753178641200066,
+      "searched_error": 0.00870854314416647,
+      "improvement": 4.463549703359604e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.33.attn.out_proj",
+      "block": 33,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009544803760945797,
+      "searched_error": 0.009460123255848885,
+      "improvement": 8.468050509691238e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.33.mlp.fc1",
+      "block": 33,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00878031924366951,
+      "searched_error": 0.008725587278604507,
+      "improvement": 5.473196506500244e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.33.mlp.fc2",
+      "block": 33,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009336614981293678,
+      "searched_error": 0.009038848802447319,
+      "improvement": 0.00029776617884635925,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.34.attn.qkv_proj",
+      "block": 34,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00897362269461155,
+      "searched_error": 0.008926293812692165,
+      "improvement": 4.7328881919384e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.34.attn.out_proj",
+      "block": 34,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009532013908028603,
+      "searched_error": 0.009444000199437141,
+      "improvement": 8.801370859146118e-05,
+      "clipped_row_fraction": 0.34375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.34.mlp.fc1",
+      "block": 34,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008846458047628403,
+      "searched_error": 0.00876232236623764,
+      "improvement": 8.413568139076233e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.34.mlp.fc2",
+      "block": 34,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009401639923453331,
+      "searched_error": 0.009106581099331379,
+      "improvement": 0.00029505882412195206,
+      "clipped_row_fraction": 0.859375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.35.attn.qkv_proj",
+      "block": 35,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00885989610105753,
+      "searched_error": 0.008797014132142067,
+      "improvement": 6.28819689154625e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.35.attn.out_proj",
+      "block": 35,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009578592143952847,
+      "searched_error": 0.00948287919163704,
+      "improvement": 9.571295231580734e-05,
+      "clipped_row_fraction": 0.328125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.35.mlp.fc1",
+      "block": 35,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008819864131510258,
+      "searched_error": 0.008781999349594116,
+      "improvement": 3.786478191614151e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.35.mlp.fc2",
+      "block": 35,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009462808258831501,
+      "searched_error": 0.009170095436275005,
+      "improvement": 0.00029271282255649567,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.36.attn.qkv_proj",
+      "block": 36,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008869685232639313,
+      "searched_error": 0.008822268806397915,
+      "improvement": 4.741642624139786e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.36.attn.out_proj",
+      "block": 36,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010005505755543709,
+      "searched_error": 0.009918516501784325,
+      "improvement": 8.698925375938416e-05,
+      "clipped_row_fraction": 0.28125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.36.mlp.fc1",
+      "block": 36,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008894680067896843,
+      "searched_error": 0.008845840580761433,
+      "improvement": 4.883948713541031e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.36.mlp.fc2",
+      "block": 36,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00928464811295271,
+      "searched_error": 0.009009410627186298,
+      "improvement": 0.00027523748576641083,
+      "clipped_row_fraction": 0.8125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.37.attn.qkv_proj",
+      "block": 37,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008858516812324524,
+      "searched_error": 0.008785505779087543,
+      "improvement": 7.301103323698044e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.37.attn.out_proj",
+      "block": 37,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010057258419692516,
+      "searched_error": 0.009970421902835369,
+      "improvement": 8.683651685714722e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.37.mlp.fc1",
+      "block": 37,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008871573954820633,
+      "searched_error": 0.008826128207147121,
+      "improvement": 4.5445747673511505e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.37.mlp.fc2",
+      "block": 37,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009507856331765652,
+      "searched_error": 0.009210675954818726,
+      "improvement": 0.0002971803769469261,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.38.attn.qkv_proj",
+      "block": 38,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008879483677446842,
+      "searched_error": 0.008825055323541164,
+      "improvement": 5.4428353905677795e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.38.attn.out_proj",
+      "block": 38,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010883932933211327,
+      "searched_error": 0.0107962591573596,
+      "improvement": 8.767377585172653e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.38.mlp.fc1",
+      "block": 38,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00895624328404665,
+      "searched_error": 0.00889838207513094,
+      "improvement": 5.786120891571045e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.38.mlp.fc2",
+      "block": 38,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009250178001821041,
+      "searched_error": 0.008992554619908333,
+      "improvement": 0.0002576233819127083,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.39.attn.qkv_proj",
+      "block": 39,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008887657895684242,
+      "searched_error": 0.008838411420583725,
+      "improvement": 4.924647510051727e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.39.attn.out_proj",
+      "block": 39,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009882594458758831,
+      "searched_error": 0.009741849265992641,
+      "improvement": 0.00014074519276618958,
+      "clipped_row_fraction": 0.46875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.39.mlp.fc1",
+      "block": 39,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008798052556812763,
+      "searched_error": 0.008744299411773682,
+      "improvement": 5.3753145039081573e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.39.mlp.fc2",
+      "block": 39,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009530093520879745,
+      "searched_error": 0.009255670011043549,
+      "improvement": 0.0002744235098361969,
+      "clipped_row_fraction": 0.765625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.40.attn.qkv_proj",
+      "block": 40,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008775589056313038,
+      "searched_error": 0.008701545186340809,
+      "improvement": 7.4043869972229e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.40.attn.out_proj",
+      "block": 40,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009700240567326546,
+      "searched_error": 0.009606442414224148,
+      "improvement": 9.379815310239792e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.40.mlp.fc1",
+      "block": 40,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008850575424730778,
+      "searched_error": 0.008782174438238144,
+      "improvement": 6.840098649263382e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.40.mlp.fc2",
+      "block": 40,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00927836075425148,
+      "searched_error": 0.009013586677610874,
+      "improvement": 0.0002647740766406059,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.41.attn.qkv_proj",
+      "block": 41,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008886943571269512,
+      "searched_error": 0.008859841153025627,
+      "improvement": 2.710241824388504e-05,
+      "clipped_row_fraction": 0.125,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.41.attn.out_proj",
+      "block": 41,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009859757497906685,
+      "searched_error": 0.009754279628396034,
+      "improvement": 0.00010547786951065063,
+      "clipped_row_fraction": 0.359375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.41.mlp.fc1",
+      "block": 41,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008647102862596512,
+      "searched_error": 0.008588128723204136,
+      "improvement": 5.8974139392375946e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.41.mlp.fc2",
+      "block": 41,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009476621635258198,
+      "searched_error": 0.009172627702355385,
+      "improvement": 0.00030399393290281296,
+      "clipped_row_fraction": 0.890625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.42.attn.qkv_proj",
+      "block": 42,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008771860040724277,
+      "searched_error": 0.00870048813521862,
+      "improvement": 7.13719055056572e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.42.attn.out_proj",
+      "block": 42,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009836846962571144,
+      "searched_error": 0.009762815199792385,
+      "improvement": 7.4031762778759e-05,
+      "clipped_row_fraction": 0.265625,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.42.mlp.fc1",
+      "block": 42,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008824005722999573,
+      "searched_error": 0.008773360401391983,
+      "improvement": 5.064532160758972e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.42.mlp.fc2",
+      "block": 42,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009384780190885067,
+      "searched_error": 0.009103096090257168,
+      "improvement": 0.00028168410062789917,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.43.attn.qkv_proj",
+      "block": 43,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008804121054708958,
+      "searched_error": 0.008760816417634487,
+      "improvement": 4.330463707447052e-05,
+      "clipped_row_fraction": 0.15625,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.43.attn.out_proj",
+      "block": 43,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010491183958947659,
+      "searched_error": 0.010409677401185036,
+      "improvement": 8.150655776262283e-05,
+      "clipped_row_fraction": 0.296875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.43.mlp.fc1",
+      "block": 43,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008889754302799702,
+      "searched_error": 0.00882685650140047,
+      "improvement": 6.289780139923096e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.43.mlp.fc2",
+      "block": 43,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00917684007436037,
+      "searched_error": 0.008919608779251575,
+      "improvement": 0.00025723129510879517,
+      "clipped_row_fraction": 0.78125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.44.attn.qkv_proj",
+      "block": 44,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008927070535719395,
+      "searched_error": 0.008864369243383408,
+      "improvement": 6.270129233598709e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.44.attn.out_proj",
+      "block": 44,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.01023987215012312,
+      "searched_error": 0.010153288953006268,
+      "improvement": 8.65831971168518e-05,
+      "clipped_row_fraction": 0.3125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.44.mlp.fc1",
+      "block": 44,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008789545856416225,
+      "searched_error": 0.008748597465455532,
+      "improvement": 4.094839096069336e-05,
+      "clipped_row_fraction": 0.140625,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.44.mlp.fc2",
+      "block": 44,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009537559002637863,
+      "searched_error": 0.009217891842126846,
+      "improvement": 0.00031966716051101685,
+      "clipped_row_fraction": 0.90625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.45.attn.qkv_proj",
+      "block": 45,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008663962595164776,
+      "searched_error": 0.008605223149061203,
+      "improvement": 5.8739446103572845e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.45.attn.out_proj",
+      "block": 45,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010613594204187393,
+      "searched_error": 0.010560794733464718,
+      "improvement": 5.2799470722675323e-05,
+      "clipped_row_fraction": 0.171875,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.45.mlp.fc1",
+      "block": 45,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008878356777131557,
+      "searched_error": 0.008829115889966488,
+      "improvement": 4.924088716506958e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.45.mlp.fc2",
+      "block": 45,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009541809558868408,
+      "searched_error": 0.009230870753526688,
+      "improvement": 0.0003109388053417206,
+      "clipped_row_fraction": 0.890625,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.46.attn.qkv_proj",
+      "block": 46,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008875416591763496,
+      "searched_error": 0.008823243901133537,
+      "improvement": 5.2172690629959106e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.46.attn.out_proj",
+      "block": 46,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010151972994208336,
+      "searched_error": 0.010055840946733952,
+      "improvement": 9.613204747438431e-05,
+      "clipped_row_fraction": 0.34375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.46.mlp.fc1",
+      "block": 46,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008902542293071747,
+      "searched_error": 0.008869115263223648,
+      "improvement": 3.3427029848098755e-05,
+      "clipped_row_fraction": 0.125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.46.mlp.fc2",
+      "block": 46,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00948946550488472,
+      "searched_error": 0.009184555150568485,
+      "improvement": 0.0003049103543162346,
+      "clipped_row_fraction": 0.875,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.47.attn.qkv_proj",
+      "block": 47,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008789182640612125,
+      "searched_error": 0.008733595721423626,
+      "improvement": 5.558691918849945e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.47.attn.out_proj",
+      "block": 47,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010352781042456627,
+      "searched_error": 0.01026771031320095,
+      "improvement": 8.507072925567627e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.47.mlp.fc1",
+      "block": 47,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008751441724598408,
+      "searched_error": 0.008704815059900284,
+      "improvement": 4.662666469812393e-05,
+      "clipped_row_fraction": 0.21875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.47.mlp.fc2",
+      "block": 47,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00933810044080019,
+      "searched_error": 0.009066533297300339,
+      "improvement": 0.0002715671434998512,
+      "clipped_row_fraction": 0.828125,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.48.attn.qkv_proj",
+      "block": 48,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00888779480010271,
+      "searched_error": 0.00883910246193409,
+      "improvement": 4.869233816862106e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.48.attn.out_proj",
+      "block": 48,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009902337566018105,
+      "searched_error": 0.009842359460890293,
+      "improvement": 5.997810512781143e-05,
+      "clipped_row_fraction": 0.234375,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.48.mlp.fc1",
+      "block": 48,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008990074507892132,
+      "searched_error": 0.008957264944911003,
+      "improvement": 3.280956298112869e-05,
+      "clipped_row_fraction": 0.125,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.48.mlp.fc2",
+      "block": 48,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.00943906232714653,
+      "searched_error": 0.00914239976555109,
+      "improvement": 0.0002966625615954399,
+      "clipped_row_fraction": 0.84375,
+      "bf16_delta_bytes": 77048760
+    },
+    {
+      "layer": "blocks.49.attn.qkv_proj",
+      "block": 49,
+      "kind": "attn.qkv_proj",
+      "shape": [
+        21504,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008817118592560291,
+      "searched_error": 0.00875457189977169,
+      "improvement": 6.254669278860092e-05,
+      "clipped_row_fraction": 0.25,
+      "bf16_delta_bytes": 115519416
+    },
+    {
+      "layer": "blocks.49.attn.out_proj",
+      "block": 49,
+      "kind": "attn.out_proj",
+      "shape": [
+        5376,
+        7168
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.010820906609296799,
+      "searched_error": 0.010764333419501781,
+      "improvement": 5.657318979501724e-05,
+      "clipped_row_fraction": 0.203125,
+      "bf16_delta_bytes": 38513592
+    },
+    {
+      "layer": "blocks.49.mlp.fc1",
+      "block": 49,
+      "kind": "mlp.fc1",
+      "shape": [
+        28672,
+        5376
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.008758306503295898,
+      "searched_error": 0.008717977441847324,
+      "improvement": 4.0329061448574066e-05,
+      "clipped_row_fraction": 0.1875,
+      "bf16_delta_bytes": 154025912
+    },
+    {
+      "layer": "blocks.49.mlp.fc2",
+      "block": 49,
+      "kind": "mlp.fc2",
+      "shape": [
+        5376,
+        14336
+      ],
+      "sample_rows": 64,
+      "absmax_error": 0.009797228500247002,
+      "searched_error": 0.009535545483231544,
+      "improvement": 0.00026168301701545715,
+      "clipped_row_fraction": 0.71875,
+      "bf16_delta_bytes": 77048760
+    }
+  ]
+}

reports/validation_ref2va.json

@@ -0,0 +1,53 @@
+{
+  "release_date": "2026-08-04",
+  "checkpoint": "Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors",
+  "bytes": 22484075856,
+  "gib": 20.93992741405964,
+  "sha256": "f4b2a8b4a53446f5d696a39deb693b09ac7ce37013793a8b19340644e38b154f",
+  "tensor_keys": 872,
+  "dtype_key_counts": {
+    "F32": 282,
+    "BF16": 250,
+    "I8": 170,
+    "U8": 170
+  },
+  "checks": {
+    "safetensors_open": "pass",
+    "int8_quant_blobs": "170/170, ConvRot group size 256",
+    "bf16_main_policy": "30/30 present; independently selected Ref2VA policy exactly matches FL2VA",
+    "nonquant_source_copies": "258/258 tensors, 4146503680 bytes, bitwise equal to raw Ref2VA HF",
+    "adaln_table": "FP32 [4097, 16]; independent rebuild max absolute difference 4.547473508864641e-13",
+    "midpoint_coordinate_relative_error": 2.1027385628258344e-6,
+    "adaln_relative_error_19_timesteps": {
+      "block_0": 3.9349262692667253e-7,
+      "block_24": 4.4643704200098e-7,
+      "block_49": 3.958584784413688e-7,
+      "final": 3.423421333081933e-7
+    },
+    "official_pruned_adaln_relative_error_19_timesteps": {
+      "block_0": 1.9679135584738106e-4,
+      "block_24": 2.090581983793527e-4,
+      "block_49": 2.110520872520283e-4,
+      "final": 1.9624050764832646e-4
+    },
+    "comfyui_cpu_load": "pass on clean unmodified ComfyUI: ModelPatcher -> MiniMaxH3Model; table [4097,16]; no time_embedder",
+    "comfyui_tested_commit": "14b05228cef127ce529bc0c08660770d4af3e9a8"
+  },
+  "sampled_main_matrix_proxy": {
+    "method": "deterministic 64 output rows per matrix, all input columns, 200/200 main matrices",
+    "official_absmax_mean_relative_error_percent": 0.934510859195143,
+    "quality21_effective_mean_relative_error_percent": 0.7639428460970521,
+    "relative_reduction_percent": 18.25211675389137,
+    "selected_bf16_official_proxy_mean_percent": 1.0683710873126984,
+    "remaining_int8_absmax_mean_percent": 0.9108884659979274,
+    "remaining_int8_searched_scale_mean_percent": 0.8987562895259437,
+    "remaining_int8_relative_reduction_percent": 1.3319058177657648,
+    "warning": "This is a weight-reconstruction proxy, not an end-to-end generation or perceptual benchmark."
+  },
+  "not_yet_tested": [
+    "GPU load through ComfyUI",
+    "peak VRAM at a target resolution/frame count",
+    "end-to-end Ref2VA generation",
+    "same-seed visual/audio A/B against BF16 and official pruned checkpoints"
+  ]
+}