Upload USAGE.md with huggingface_hub
Browse files
USAGE.md
CHANGED
|
@@ -108,7 +108,7 @@ sh ./opencoti-llamafile-0.10.3-c3-x86_64.llamafile --server --port 8080 \
|
|
| 108 |
|---|---:|---|---|
|
| 109 |
| `…-x86_64.llamafile` | ~4.8 GB | Linux/macOS/BSD, x86_64 + aarch64 | CUDA DSO **embedded** (x86_64-linux, sm_75/80/86/89/90/120f); self-extracts on first `-ngl` run |
|
| 110 |
| `…-win-x86_64.llamafile.exe` | ~55 MB | **Windows** + Linux/macOS/BSD, x86_64 + aarch64 | none embedded — CPU works everywhere out of the box; GPU via side-load (below) |
|
| 111 |
-
| `…-aarch64.llamafile`
|
| 112 |
|
| 113 |
The host binary inside every variant is byte-for-byte the same APE
|
| 114 |
(same patches, same features); they differ only in the embedded GPU
|
|
@@ -134,6 +134,10 @@ to `.exe` (Windows needs the extension) can.
|
|
| 134 |
cp ggml-cuda-x86_64.so ~/.llamafile/v/0.10.3/ggml-cuda.so
|
| 135 |
```
|
| 136 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 137 |
- **No CUDA available:** nothing to do — it falls back to CPU
|
| 138 |
inference automatically.
|
| 139 |
|
|
|
|
| 108 |
|---|---:|---|---|
|
| 109 |
| `…-x86_64.llamafile` | ~4.8 GB | Linux/macOS/BSD, x86_64 + aarch64 | CUDA DSO **embedded** (x86_64-linux, sm_75/80/86/89/90/120f); self-extracts on first `-ngl` run |
|
| 110 |
| `…-win-x86_64.llamafile.exe` | ~55 MB | **Windows** + Linux/macOS/BSD, x86_64 + aarch64 | none embedded — CPU works everywhere out of the box; GPU via side-load (below) |
|
| 111 |
+
| `…-aarch64.llamafile` | ~1.9 GB | same platforms | CUDA sbsa DSO **embedded** (aarch64-linux, sm_110f DGX Spark GB10 / Jetson Thor + sm_121a); GPU out of the box on aarch64-linux CUDA hosts |
|
| 112 |
|
| 113 |
The host binary inside every variant is byte-for-byte the same APE
|
| 114 |
(same patches, same features); they differ only in the embedded GPU
|
|
|
|
| 134 |
cp ggml-cuda-x86_64.so ~/.llamafile/v/0.10.3/ggml-cuda.so
|
| 135 |
```
|
| 136 |
|
| 137 |
+
- **Linux aarch64 (DGX Spark / GB10 class):** same mechanism with the
|
| 138 |
+
sbsa DSO — `dso/<ver>-<tag>/ggml-cuda-sbsa-aarch64.so` →
|
| 139 |
+
`~/.llamafile/v/0.10.3/ggml-cuda.so`. Or just download the
|
| 140 |
+
`…-aarch64.llamafile` artifact, which has it embedded.
|
| 141 |
- **No CUDA available:** nothing to do — it falls back to CPU
|
| 142 |
inference automatically.
|
| 143 |
|