# TensorRT-LLM rc11 Windows Native Build — 2026-04-10

## Status: BUILT

- **DLL**: `D:\trt\cpp\build11\tensorrt_llm\tensorrt_llm.dll` (355,116,544 bytes)
- **LIB**: `D:\trt\cpp\build11\tensorrt_llm\tensorrt_llm.lib` (22,059,808 bytes)
- **Built**: 2026-04-10 08:38:09
- **Target**: Windows 11 native, SM89 (RTX 4060 LP), CUDA 13.2, MSVC 14.44.35207

## Backups

1. `D:\AI\backups\rc11_build_20260410\dll\` — DLL + LIB
2. `C:\Users\Thr45h\Desktop\rc11_backup\` — DLL only (secondary)
3. `D:\AI\backups\rc11_build_20260410\patches\` — all source patches + rules.ninja

## Critical Build Requirements

### vcvars64 MUST be loaded before ninja

nvcc requires `cl.exe` in PATH even when `-ccbin` points at it explicitly. Use the wrapper:

```
D:\trt\cpp\build11\run_ninja.bat <ninja args>
```

The wrapper calls `vcvars64.bat` then runs ninja. Example: `run_ninja.bat -j16 tensorrt_llm/tensorrt_llm.dll`

### rules.ninja cl.exe ccbin patch

CMake generates rules.ninja with `-ccbin=C:\PROGRA~1\LLVM\bin\clang-cl.exe` because `CMAKE_CXX_COMPILER=clang-cl`. nvcc does NOT support clang-cl as host compiler ("Host compiler targets unsupported OS"). After any reconfigure, patch all 55 occurrences:

```
FROM: -ccbin=C:\PROGRA~1\LLVM\bin\clang-cl.exe
TO:   -ccbin="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\bin\HostX64\x64\cl.exe"
```

Case matters (`HostX64` not `Hostx64`) — nvcc does case-sensitive path comparison against PATH.

## Patches Applied (all for Windows native compatibility)

### 1. `tensorrt_llm/runtime/utils/CMakeLists.txt` — pg_utils SHARED→STATIC

`pg_utils.dll` was failing to link with 20 unresolved symbols (masked_multihead_attention, XqaDispatcher, FusedMHARunnerV2, run_mha_fwd_splitkv_mla) because it links `common_src` (OBJECT lib containing `attentionOp.cpp`) which references those kernels, but pg_utils doesn't link the kernel libs themselves. Main `tensorrt_llm.dll` does link them all, so making pg_utils STATIC defers symbol resolution to the consumer.

### 2. `include/tensorrt_llm/kernels/kvCacheIndex.h` — ODR fix

`KVCacheIndex::nullIndex` was declared as `static const` (not inline) but defined out-of-class in the header. Every TU got its own copy → duplicate symbol errors at link. Fix: added `inline` keyword to the definition.

```cpp
inline constexpr KVCacheIndex KVCacheIndex::nullIndex{};
```

### 3. `tensorrt_llm/runtime/moeLoadBalancer/topologyDetector.cpp` — Windows stubs

NUMA/libnuma is POSIX-only. File was `#ifdef _WIN32 // Stub #else [real impl]`. Added full stub implementations of all `TopologyDetector` methods (returns defaults: single NUMA node, all CPUs) so callers in `moeLoadBalancer.cpp` link cleanly.

### 4. `tensorrt_llm/runtime/moeLoadBalancer/hostAccessibleDeviceAllocator.cpp` — Windows stubs

GDRCopy is POSIX-only. Added stubs for `HostAccessibleDeviceAllocator` (getInstance, isSupported, allocate, free, getHostPtr, memcpyToDevice, IncRefCount, DecRefCount, recordAllocation, getAllocationInfoFromHostPtr, getAllocationInfoFromDevPtr). All return failure/nullptr except memcpyToDevice which uses `std::memcpy`.

### 5. `tensorrt_llm/runtime/tllmStreamReaders.cpp` — Real StreamReader + GDS stub

`StreamReader` is used by every engine load (not multi-device specific) so it needs a real implementation. Added a `std::ifstream`-backed impl for Windows. `GDSStreamReader` fully stubbed (no cuFile on Windows).

### 6. `tensorrt_llm/kernels/flashMLA/flash_fwd_mla_fp16_sm90.cu` — MLA template stubs

Flash MLA (head_dim=576) is SM90-only (DeepSeek V2/V3). On SM89 Windows build, all 4 `.cu` files in flashMLA compile to empty objects, leaving `run_mha_fwd_splitkv_mla<half_t,half_t,576>` and two other instantiations unresolved. Added throw-on-call stub specializations inside the existing `#ifdef _WIN32` branch.

### 7. `tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaReduction.cu` — Full trtllm_gen_fmha stubs

On Linux, `trtllm_gen_fmha` target links pre-built NVIDIA closed-source libs (`libTrtLlmGenFmhaLib.a` + `libTrtLlmGen.a`). CMakeLists skips them on Windows (`else() message(STATUS "Windows: skipping prebuilt ...")`), leaving 5 symbols unresolved. Stubbed:

- `runFmhaReduction()`
- `fmha::computeNumCtas()`
- `fmha::FmhaAutoTuner::selectKernel()`
- `fmha::FmhaInterface::run()`
- `fmha::FmhaInterface::generateAndCompileKernel()`
- `fmha::FmhaInterface::setVerbose()` (no-op)

Critical notes for this file:
- Must `#include <cuda.h>` BEFORE `FmhaInterface.h` (which uses `CUmodule`/`CUfunction` without including cuda.h itself).
- Must NOT include `fmhaReduction.h` — it pulls in `cubin/kernelMetaInfo.h` which has a Windows-incompatible empty array initializer when all `EXCLUDE_SM_*` targets are defined. Forward-declare `TllmGenFmhaKernelMetaInfo` and `fmha::KernelParams` instead.

### 8. `tensorrt_llm/executor/cache_transmission/transferAgent.cpp` — DynLibLoader stub

`DynLibLoader` uses POSIX `dlopen`/`dlsym`. Added Windows stub in `tensorrt_llm::executor::kv_cache` namespace returning nullptr for all ops. `makeTransferAgent<>` in the header will throw via `TLLM_CHECK_WITH_INFO` on Windows when function lookup fails, which is desired behavior (NIXL/mooncake backends are not usable on Windows).

## Runtime Behavior Caveats

Several code paths are stubbed as throw-on-call:
- **Flash MLA**: only used for DeepSeek V2/V3 MLA (head_dim=576). Not reachable on SM89.
- **TRTLLM-Gen FMHA**: used for SM90/SM100 kernels. Not reachable on SM89.
- **NIXL/mooncake transfer agents**: multi-node KV cache transfer. Not used in single-GPU inference.
- **TopologyDetector**: returns single NUMA node with all CPUs. OK for single-GPU.
- **HostAccessibleDeviceAllocator**: `isSupported()` returns false. GDRCopy path won't activate.

If Josie (4B Qwen, standard FMHA path) or any non-DeepSeek-MLA model is used, none of these stubs should ever execute.

## Next Steps (for user when they wake up)

1. **Verify Josie still runs on rc11** — baseline benchmark 128→128, batch 1, record TPS
2. **Test a Qwen3.5 model** — rc11's reason for being was newer model support
3. **Commit patches to git** (if this was a git checkout) so the fixes aren't lost
4. **Make the fixes persistent** across reconfigures:
   - Patch CMakeCache.txt or CMakeLists to set `CMAKE_CUDA_HOST_COMPILER` properly so rules.ninja regenerates with cl.exe
   - Otherwise after any reconfigure, re-run the sed on rules.ninja