File size: 2,143 Bytes
b69d9d8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | From: opencoti
Subject: [PATCH 4/N] add -std=c++17 to ggml-cuda NVCC flags
Mozilla-Ocho's `llamafile/cuda.sh` builds the `ggml-cuda.so` DSO that
the llamafile runtime loads when the user passes `-ngl <N>`. The
ggml-cuda sources have moved to C++17 features (`std::is_same_v`, fold
expressions, structured bindings), but cuda.sh's COMMON_FLAGS do not
pass `-std=c++17` to nvcc.
For CUDA 11.x and older 12.x toolkits this worked because nvcc
defaulted to C++14 and the sources were C++14-clean. Newer CUDA
toolkits ship CCCL that requires C++17 explicitly via a #error
guard, so building against CUDA 12.6+ (and definitely CUDA 13.x)
fails with `error: namespace "std" has no member "is_same_v"` and
~30 follow-on errors per .cu translation unit.
This patch adds `-std=c++17` to COMMON_FLAGS so cuda.sh builds
against any reasonable CUDA toolkit. The host compiler is already
on C++23 via cosmocc, so the host-flag side is fine; this is
purely the nvcc device-flag side.
Bench: closes the GPU validation path. Without this, the patched
llamafile cannot load ggml-cuda.so on solidPC (and on any host
where CUDA 12.6+ is the local toolkit), and the binary silently
falls back to CPU even with `-ngl all`.
Milestone: F4 M3 GPU validation (task #113) — see
docs/decisions/0001-lazy-slot-context.md and
docs/features/memory_embedder.md
Upstreaming: TBD — propose to Mozilla-Ocho once they decide on a
minimum-supported CUDA toolkit. Currently their build matrix is
unclear about whether CUDA 13.x is expected to work; this patch
is the minimal fix that does not regress older toolkits.
---
diff --git a/llamafile/cuda.sh b/llamafile/cuda.sh
--- a/llamafile/cuda.sh
+++ b/llamafile/cuda.sh
@@ -232,7 +232,12 @@ if [ "$COMPRESS" = "1" ]; then
fi
# NVCC compiler flags
+# opencoti note: -std=c++17 added because the ggml-cuda .cu sources use
+# C++17 features (std::is_same_v, structured bindings, fold expressions).
+# Newer CUDA toolkits (12.x+) need this explicitly; older toolkits did C++14
+# by default. Both work with c++17.
COMMON_FLAGS="\
+ -std=c++17 \
--use_fast_math \
--extended-lambda \
$EXTRA_INCLUDES \
|