GrEarl commited on
Commit
e6592bd
·
verified ·
1 Parent(s): ef7d97a

finalize pinned NVFP4A16 checkpoint metadata

Browse files
Files changed (2) hide show
  1. README.md +32 -14
  2. nvfp4_conversion_manifest.json +15 -3
README.md CHANGED
@@ -41,26 +41,43 @@ calibration and quality-validation run.
41
 
42
  ## Runtime validation (2026-07-27 UTC)
43
 
44
- A purpose-built **vLLM one-layer routed-MoE smoke test passed** for this
45
- published artifact. The harness used vLLM's official compressed-tensors
46
- NVFP4A16 parameter layout, post-load conversion, fused-MoE method, and SiTU
47
- activation; it was not the full vLLM model loader or full K3 graph.
 
 
 
48
 
49
- - Evidence run: `20260727T220141Z`
 
50
  - Tested artifact revision: `8c22f32993e94e063cb634a03b7fc2f9ff539621`
51
  - Checkpoint shard: `model-00049-of-000096.safetensors` (17,916,197,160 bytes)
52
  - Runtime image: `vllm/vllm-openai:kimi-k3`
53
  - Pinned linux/amd64 image: `vllm/vllm-openai@sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e`
54
  - vLLM: `0.1.dev19262+gb6bbf29dd.d20260727`
55
  - Hardware: NVIDIA B200, SM100; torch `2.13.0+cu130`, CUDA 13.0
56
- - Exact MoE shape: 896 routed experts, top-k 16, `3584 -> 3072 -> 3584`, 2 tokens
57
- - Loaded tensors: 8,064 (`U8=2,688`, `F8_E4M3=2,688`, `F32=2,688`); all 896 experts
 
 
 
58
  - SiTU: beta `4.0`, linear beta `25.0`
59
  - Backend: `MARLIN` / `MarlinExperts`
60
- - Observed operators: `_moe_C::moe_wna16_marlin_gemm` (2 calls), Marlin CUDA
61
- kernel, `_C::situ_and_mul`, `vllm::situ_and_mul_kernel`, routing alignment,
62
- and MoE sum
63
- - Output: shape `[2, 3584]`, finite and nonzero
 
 
 
 
 
 
 
 
 
 
64
 
65
  **SGLang runtime was not executed.** A zero-cost gate against the official
66
  `lmsysorg/sglang:kimi-k3` image (linux/amd64 digest
@@ -71,9 +88,10 @@ FLOAT tensor-group MoE path for `input_activations: null`. The gate therefore
71
  returned `BLOCKED / DO_NOT_RUN_GPU` (`20260727T221943Z`) before weight download
72
  or GPU allocation.
73
 
74
- Not verified: the full 93-layer / 2.8T model, KDA/MLA/vision integration,
75
- all 96 shards loaded together, end-to-end token generation, serving quality,
76
- or SGLang load/forward/kernel dispatch.
 
77
 
78
  ## License
79
 
 
41
 
42
  ## Runtime validation (2026-07-27 UTC)
43
 
44
+ A **complete Stable LatentMoE block passed on vLLM for this published
45
+ artifact** using the official `KimiMoE` class. The test first loaded and ran the
46
+ block through a purpose-built layout path, then rebuilt it and loaded the same
47
+ 8,072 source tensors through the real K3 model-class chain:
48
+ `KimiK3ForConditionalGeneration.load_weights` -> `AutoWeightsLoader` ->
49
+ `KimiLinearForCausalLM.load_weights` -> `KimiLinearModel.load_weights` -> the
50
+ generic `process_weights_after_loading` traversal.
51
 
52
+ - Evidence run: `20260727T234215Z`
53
+ - Earlier routed-expert-core run: `20260727T220141Z`
54
  - Tested artifact revision: `8c22f32993e94e063cb634a03b7fc2f9ff539621`
55
  - Checkpoint shard: `model-00049-of-000096.safetensors` (17,916,197,160 bytes)
56
  - Runtime image: `vllm/vllm-openai:kimi-k3`
57
  - Pinned linux/amd64 image: `vllm/vllm-openai@sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e`
58
  - vLLM: `0.1.dev19262+gb6bbf29dd.d20260727`
59
  - Hardware: NVIDIA B200, SM100; torch `2.13.0+cu130`, CUDA 13.0
60
+ - Complete path: FP32 router + correction bias, `7168 -> 3584` latent down,
61
+ 896 routed experts with real top-k 16 and `3584 -> 3072 -> 3584`, routed
62
+ RMSNorm + `3584 -> 7168` up, and 2 shared experts (`7168 -> 6144 -> 7168`)
63
+ - Loaded source tensors: 8,072 = 8 nonexpert block tensors + 8,064 routed-expert
64
+ quant tensors; all 896 experts
65
  - SiTU: beta `4.0`, linear beta `25.0`
66
  - Backend: `MARLIN` / `MarlinExperts`
67
+ - Observed operators: `vllm::moe_forward_shared`,
68
+ `_moe_C::moe_wna16_marlin_gemm`, `_moe_C::grouped_topk`,
69
+ `_C::situ_and_mul`, `vllm_ir::rms_norm`, routing alignment, and MoE sum
70
+ - Router logits: `[2, 896]`; selected IDs `[2, 16]`; 16 unique in-range
71
+ experts per token; finite normalized weights
72
+ - Both direct and real model-class-loader outputs: `[2, 7168]`, finite and
73
+ nonzero; sampled parameter fingerprints matched exactly and outputs were
74
+ bit-exact (`max_abs_difference=0`)
75
+ - Complete-block validation run cost: `$0.341178`; all attempts including the
76
+ externally canceled run: `$3.892933 < $4.5`
77
+
78
+ This verifies the real K3 outer/inner `load_weights` mapping and generic
79
+ post-load traversal on a reduced one-block wrapper. `DefaultModelLoader`
80
+ index/file traversal was not exercised, and the full model was not constructed.
81
 
82
  **SGLang runtime was not executed.** A zero-cost gate against the official
83
  `lmsysorg/sglang:kimi-k3` image (linux/amd64 digest
 
88
  returned `BLOCKED / DO_NOT_RUN_GPU` (`20260727T221943Z`) before weight download
89
  or GPU allocation.
90
 
91
+ Not verified: the full 93-layer / 2.8T model, all 96 shards resident together,
92
+ `DefaultModelLoader` index/file traversal, KDA/MLA/vision integration, TP16,
93
+ end-to-end token generation, serving quality, or SGLang load/forward/kernel
94
+ dispatch.
95
 
96
  ## License
97
 
nvfp4_conversion_manifest.json CHANGED
@@ -19,14 +19,26 @@
19
  "tensor_bytes": 1645946964992,
20
  "lossless_over_span_policy": "fail; never silently requantize",
21
  "runtime_validation": {
22
- "level": "vllm_one_layer_routed_moe",
23
  "vllm_verified": true,
24
- "vllm_run_id": "20260727T220141Z",
 
25
  "vllm_image_amd64_digest": "sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e",
 
 
 
 
 
 
 
 
 
 
26
  "sglang_verified": false,
27
  "sglang_static_gate": "BLOCKED",
28
  "sglang_static_run_id": "20260727T221943Z",
29
  "full_model_load_verified": false,
30
- "full_generation_verified": false
 
31
  }
32
  }
 
19
  "tensor_bytes": 1645946964992,
20
  "lossless_over_span_policy": "fail; never silently requantize",
21
  "runtime_validation": {
22
+ "level": "vllm_complete_stable_latent_moe_reduced_model_class_loader",
23
  "vllm_verified": true,
24
+ "vllm_run_id": "20260727T234215Z",
25
+ "vllm_routed_expert_core_run_id": "20260727T220141Z",
26
  "vllm_image_amd64_digest": "sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e",
27
+ "complete_stable_latent_moe_verified": true,
28
+ "real_k3_load_weights_mapping_verified": true,
29
+ "generic_post_load_traversal_verified": true,
30
+ "default_model_loader_file_traversal_verified": false,
31
+ "loader_chain": [
32
+ "KimiK3ForConditionalGeneration.load_weights",
33
+ "AutoWeightsLoader",
34
+ "KimiLinearForCausalLM.load_weights",
35
+ "KimiLinearModel.load_weights"
36
+ ],
37
  "sglang_verified": false,
38
  "sglang_static_gate": "BLOCKED",
39
  "sglang_static_run_id": "20260727T221943Z",
40
  "full_model_load_verified": false,
41
+ "full_generation_verified": false,
42
+ "tp16_verified": false
43
  }
44
  }