ManniX-ITA commited on
Commit
6e4de51
·
verified ·
1 Parent(s): 73f68fa

Upload USAGE.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. USAGE.md +5 -1
USAGE.md CHANGED
@@ -108,7 +108,7 @@ sh ./opencoti-llamafile-0.10.3-c3-x86_64.llamafile --server --port 8080 \
108
  |---|---:|---|---|
109
  | `…-x86_64.llamafile` | ~4.8 GB | Linux/macOS/BSD, x86_64 + aarch64 | CUDA DSO **embedded** (x86_64-linux, sm_75/80/86/89/90/120f); self-extracts on first `-ngl` run |
110
  | `…-win-x86_64.llamafile.exe` | ~55 MB | **Windows** + Linux/macOS/BSD, x86_64 + aarch64 | none embedded — CPU works everywhere out of the box; GPU via side-load (below) |
111
- | `…-aarch64.llamafile` *(planned)* | | same platforms | CUDA sbsa DSO embedded (sm_110f Jetson Thor, sm_121a DGX Spark GB10) |
112
 
113
  The host binary inside every variant is byte-for-byte the same APE
114
  (same patches, same features); they differ only in the embedded GPU
@@ -134,6 +134,10 @@ to `.exe` (Windows needs the extension) can.
134
  cp ggml-cuda-x86_64.so ~/.llamafile/v/0.10.3/ggml-cuda.so
135
  ```
136
 
 
 
 
 
137
  - **No CUDA available:** nothing to do — it falls back to CPU
138
  inference automatically.
139
 
 
108
  |---|---:|---|---|
109
  | `…-x86_64.llamafile` | ~4.8 GB | Linux/macOS/BSD, x86_64 + aarch64 | CUDA DSO **embedded** (x86_64-linux, sm_75/80/86/89/90/120f); self-extracts on first `-ngl` run |
110
  | `…-win-x86_64.llamafile.exe` | ~55 MB | **Windows** + Linux/macOS/BSD, x86_64 + aarch64 | none embedded — CPU works everywhere out of the box; GPU via side-load (below) |
111
+ | `…-aarch64.llamafile` | ~1.9 GB | same platforms | CUDA sbsa DSO **embedded** (aarch64-linux, sm_110f DGX Spark GB10 / Jetson Thor + sm_121a); GPU out of the box on aarch64-linux CUDA hosts |
112
 
113
  The host binary inside every variant is byte-for-byte the same APE
114
  (same patches, same features); they differ only in the embedded GPU
 
134
  cp ggml-cuda-x86_64.so ~/.llamafile/v/0.10.3/ggml-cuda.so
135
  ```
136
 
137
+ - **Linux aarch64 (DGX Spark / GB10 class):** same mechanism with the
138
+ sbsa DSO — `dso/<ver>-<tag>/ggml-cuda-sbsa-aarch64.so` →
139
+ `~/.llamafile/v/0.10.3/ggml-cuda.so`. Or just download the
140
+ `…-aarch64.llamafile` artifact, which has it embedded.
141
  - **No CUDA available:** nothing to do — it falls back to CPU
142
  inference automatically.
143