finalize pinned NVFP4A16 checkpoint metadata
Browse files- README.md +32 -14
- nvfp4_conversion_manifest.json +15 -3
README.md
CHANGED
|
@@ -41,26 +41,43 @@ calibration and quality-validation run.
|
|
| 41 |
|
| 42 |
## Runtime validation (2026-07-27 UTC)
|
| 43 |
|
| 44 |
-
A
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
-
- Evidence run: `
|
|
|
|
| 50 |
- Tested artifact revision: `8c22f32993e94e063cb634a03b7fc2f9ff539621`
|
| 51 |
- Checkpoint shard: `model-00049-of-000096.safetensors` (17,916,197,160 bytes)
|
| 52 |
- Runtime image: `vllm/vllm-openai:kimi-k3`
|
| 53 |
- Pinned linux/amd64 image: `vllm/vllm-openai@sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e`
|
| 54 |
- vLLM: `0.1.dev19262+gb6bbf29dd.d20260727`
|
| 55 |
- Hardware: NVIDIA B200, SM100; torch `2.13.0+cu130`, CUDA 13.0
|
| 56 |
-
-
|
| 57 |
-
|
|
|
|
|
|
|
|
|
|
| 58 |
- SiTU: beta `4.0`, linear beta `25.0`
|
| 59 |
- Backend: `MARLIN` / `MarlinExperts`
|
| 60 |
-
- Observed operators: `
|
| 61 |
-
|
| 62 |
-
and MoE sum
|
| 63 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
**SGLang runtime was not executed.** A zero-cost gate against the official
|
| 66 |
`lmsysorg/sglang:kimi-k3` image (linux/amd64 digest
|
|
@@ -71,9 +88,10 @@ FLOAT tensor-group MoE path for `input_activations: null`. The gate therefore
|
|
| 71 |
returned `BLOCKED / DO_NOT_RUN_GPU` (`20260727T221943Z`) before weight download
|
| 72 |
or GPU allocation.
|
| 73 |
|
| 74 |
-
Not verified: the full 93-layer / 2.8T model,
|
| 75 |
-
|
| 76 |
-
or SGLang load/forward/kernel
|
|
|
|
| 77 |
|
| 78 |
## License
|
| 79 |
|
|
|
|
| 41 |
|
| 42 |
## Runtime validation (2026-07-27 UTC)
|
| 43 |
|
| 44 |
+
A **complete Stable LatentMoE block passed on vLLM for this published
|
| 45 |
+
artifact** using the official `KimiMoE` class. The test first loaded and ran the
|
| 46 |
+
block through a purpose-built layout path, then rebuilt it and loaded the same
|
| 47 |
+
8,072 source tensors through the real K3 model-class chain:
|
| 48 |
+
`KimiK3ForConditionalGeneration.load_weights` -> `AutoWeightsLoader` ->
|
| 49 |
+
`KimiLinearForCausalLM.load_weights` -> `KimiLinearModel.load_weights` -> the
|
| 50 |
+
generic `process_weights_after_loading` traversal.
|
| 51 |
|
| 52 |
+
- Evidence run: `20260727T234215Z`
|
| 53 |
+
- Earlier routed-expert-core run: `20260727T220141Z`
|
| 54 |
- Tested artifact revision: `8c22f32993e94e063cb634a03b7fc2f9ff539621`
|
| 55 |
- Checkpoint shard: `model-00049-of-000096.safetensors` (17,916,197,160 bytes)
|
| 56 |
- Runtime image: `vllm/vllm-openai:kimi-k3`
|
| 57 |
- Pinned linux/amd64 image: `vllm/vllm-openai@sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e`
|
| 58 |
- vLLM: `0.1.dev19262+gb6bbf29dd.d20260727`
|
| 59 |
- Hardware: NVIDIA B200, SM100; torch `2.13.0+cu130`, CUDA 13.0
|
| 60 |
+
- Complete path: FP32 router + correction bias, `7168 -> 3584` latent down,
|
| 61 |
+
896 routed experts with real top-k 16 and `3584 -> 3072 -> 3584`, routed
|
| 62 |
+
RMSNorm + `3584 -> 7168` up, and 2 shared experts (`7168 -> 6144 -> 7168`)
|
| 63 |
+
- Loaded source tensors: 8,072 = 8 nonexpert block tensors + 8,064 routed-expert
|
| 64 |
+
quant tensors; all 896 experts
|
| 65 |
- SiTU: beta `4.0`, linear beta `25.0`
|
| 66 |
- Backend: `MARLIN` / `MarlinExperts`
|
| 67 |
+
- Observed operators: `vllm::moe_forward_shared`,
|
| 68 |
+
`_moe_C::moe_wna16_marlin_gemm`, `_moe_C::grouped_topk`,
|
| 69 |
+
`_C::situ_and_mul`, `vllm_ir::rms_norm`, routing alignment, and MoE sum
|
| 70 |
+
- Router logits: `[2, 896]`; selected IDs `[2, 16]`; 16 unique in-range
|
| 71 |
+
experts per token; finite normalized weights
|
| 72 |
+
- Both direct and real model-class-loader outputs: `[2, 7168]`, finite and
|
| 73 |
+
nonzero; sampled parameter fingerprints matched exactly and outputs were
|
| 74 |
+
bit-exact (`max_abs_difference=0`)
|
| 75 |
+
- Complete-block validation run cost: `$0.341178`; all attempts including the
|
| 76 |
+
externally canceled run: `$3.892933 < $4.5`
|
| 77 |
+
|
| 78 |
+
This verifies the real K3 outer/inner `load_weights` mapping and generic
|
| 79 |
+
post-load traversal on a reduced one-block wrapper. `DefaultModelLoader`
|
| 80 |
+
index/file traversal was not exercised, and the full model was not constructed.
|
| 81 |
|
| 82 |
**SGLang runtime was not executed.** A zero-cost gate against the official
|
| 83 |
`lmsysorg/sglang:kimi-k3` image (linux/amd64 digest
|
|
|
|
| 88 |
returned `BLOCKED / DO_NOT_RUN_GPU` (`20260727T221943Z`) before weight download
|
| 89 |
or GPU allocation.
|
| 90 |
|
| 91 |
+
Not verified: the full 93-layer / 2.8T model, all 96 shards resident together,
|
| 92 |
+
`DefaultModelLoader` index/file traversal, KDA/MLA/vision integration, TP16,
|
| 93 |
+
end-to-end token generation, serving quality, or SGLang load/forward/kernel
|
| 94 |
+
dispatch.
|
| 95 |
|
| 96 |
## License
|
| 97 |
|
nvfp4_conversion_manifest.json
CHANGED
|
@@ -19,14 +19,26 @@
|
|
| 19 |
"tensor_bytes": 1645946964992,
|
| 20 |
"lossless_over_span_policy": "fail; never silently requantize",
|
| 21 |
"runtime_validation": {
|
| 22 |
-
"level": "
|
| 23 |
"vllm_verified": true,
|
| 24 |
-
"vllm_run_id": "
|
|
|
|
| 25 |
"vllm_image_amd64_digest": "sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
"sglang_verified": false,
|
| 27 |
"sglang_static_gate": "BLOCKED",
|
| 28 |
"sglang_static_run_id": "20260727T221943Z",
|
| 29 |
"full_model_load_verified": false,
|
| 30 |
-
"full_generation_verified": false
|
|
|
|
| 31 |
}
|
| 32 |
}
|
|
|
|
| 19 |
"tensor_bytes": 1645946964992,
|
| 20 |
"lossless_over_span_policy": "fail; never silently requantize",
|
| 21 |
"runtime_validation": {
|
| 22 |
+
"level": "vllm_complete_stable_latent_moe_reduced_model_class_loader",
|
| 23 |
"vllm_verified": true,
|
| 24 |
+
"vllm_run_id": "20260727T234215Z",
|
| 25 |
+
"vllm_routed_expert_core_run_id": "20260727T220141Z",
|
| 26 |
"vllm_image_amd64_digest": "sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e",
|
| 27 |
+
"complete_stable_latent_moe_verified": true,
|
| 28 |
+
"real_k3_load_weights_mapping_verified": true,
|
| 29 |
+
"generic_post_load_traversal_verified": true,
|
| 30 |
+
"default_model_loader_file_traversal_verified": false,
|
| 31 |
+
"loader_chain": [
|
| 32 |
+
"KimiK3ForConditionalGeneration.load_weights",
|
| 33 |
+
"AutoWeightsLoader",
|
| 34 |
+
"KimiLinearForCausalLM.load_weights",
|
| 35 |
+
"KimiLinearModel.load_weights"
|
| 36 |
+
],
|
| 37 |
"sglang_verified": false,
|
| 38 |
"sglang_static_gate": "BLOCKED",
|
| 39 |
"sglang_static_run_id": "20260727T221943Z",
|
| 40 |
"full_model_load_verified": false,
|
| 41 |
+
"full_generation_verified": false,
|
| 42 |
+
"tp16_verified": false
|
| 43 |
}
|
| 44 |
}
|