Mask Generation
Transformers.js
ONNX
swin
onnxruntime
onnxruntime-web
webgpu
vision
image-segmentation
background-removal
salient-object-detection
matting
mvanet
Instructions to use MarcinEU/finegrain-box-segmenter-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use MarcinEU/finegrain-box-segmenter-ONNX with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('mask-generation', 'MarcinEU/finegrain-box-segmenter-ONNX');
card: escape tildes to prevent strikethrough rendering
Browse files
README.md
CHANGED
|
@@ -24,11 +24,11 @@ tags:
|
|
| 24 |
FP32 ONNX export of [**finegrain/finegrain-box-segmenter**](https://huggingface.co/finegrain/finegrain-box-segmenter) (an MVANet high-resolution background remover), ready to run in the browser, Node.js and Python with [ONNX Runtime](https://onnxruntime.ai/) (CPU, WebGPU (recommended), DirectML).
|
| 25 |
|
| 26 |
- **Base model:** `finegrain/finegrain-box-segmenter` (MIT) — MVANet, SafeTensors, [arXiv:2404.07445](https://arxiv.org/abs/2404.07445)
|
| 27 |
-
- **Architecture:** MVANet with a **Swin-B** backbone, **~94.6 M parameters**; static 1024×1024
|
| 28 |
input, batch fixed at 1.
|
| 29 |
- **What this repo adds:** the same network exported to ONNX (full-precision floating-point **fp32**), so it runs
|
| 30 |
without PyTorch/refiners.
|
| 31 |
-
- **Why fp32:** maximum compatibility with zero precision risk — the fp32 graph runs on CPU, WebGPU and DirectML, and stays **bit-identical** to the original release; at ~94.6 M parameters it is still cheap enough to run on a local machine. (An fp16 build (\~403 MB) exists and its masks matched fp32 in testing, but it is not published, to keep the release simple — open a discussion if you need it.) Regarding why this release uses the "quantized" tag: it's a fairly common convention for ONNX / format derivatives (same as e.g. `onnx-community/BiRefNet-ONNX`), so I decided to go this route for tagging.
|
| 32 |
- **Output is bit-identical to the PyTorch reference** on CPU (mask MAE `0.000 / 255`; random-input logits `max|Δ| = 1.5e-5`).
|
| 33 |
|
| 34 |
## Examples
|
|
@@ -74,11 +74,11 @@ The short version: **finegrain-box-segmenter is strongest on soft, semi-transpar
|
|
| 74 |
|
| 75 |
| File | Precision | Size | Notes |
|
| 76 |
|------|-----------|-----:|-------|
|
| 77 |
-
| `onnx/model.onnx` | fp32 | ~805 MB | full precision; CPU + WebGPU; bit-identical parity with PyTorch |
|
| 78 |
|
| 79 |
-
ONNX operator set version 17, ~94.6 M parameters, SHA256(onnx/model.onnx) = `d3f60fc778f2b14435a8df3d61511c293cec3a88c670ad8ecac5035c6b9730d5`
|
| 80 |
|
| 81 |
-
> Why 805 MB for a 94.6 M-param model: ~378 MB is fp32 weights; the other ~425 MB is constant-folded shifted-window (Swin) attention masks / position embeddings baked in at export (`do_constant_folding=True`). Note that disabling the folding will make the file *larger*, not smaller.
|
| 82 |
|
| 83 |
## I/O contract
|
| 84 |
|
|
@@ -179,7 +179,7 @@ It resizes the whole image to 1024², normalizes, runs the session, applies sigm
|
|
| 179 |
|
| 180 |
## Usage — Python (onnxruntime)
|
| 181 |
|
| 182 |
-
Verified against this exact export (`pip install onnxruntime pillow numpy huggingface_hub`); masks match the Node reference pipeline to ~1/255 mean difference:
|
| 183 |
|
| 184 |
```python
|
| 185 |
from huggingface_hub import hf_hub_download
|
|
@@ -203,7 +203,7 @@ cutout.save("cutout.png"); mask.save("mask.png")
|
|
| 203 |
|
| 204 |
## Performance across environments
|
| 205 |
|
| 206 |
-
The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **batch-5**, so it is **compute-bound and the inference time is ~constant regardless of input image size** — the source resolution only changes the negligible host-side resize (`pre` + `post`, ≈30–500 ms for 1080p→8K). The numbers below are therefore "**per image**" for any resolution. All EPs run the **same fp32 model** (no fp16/NPU path). Measured with `onnxruntime-node` 1.26, **15 runs/cell, median**.
|
| 207 |
|
| 208 |
### Measured — median inference time, seconds per image (lower is better)
|
| 209 |
|
|
@@ -212,7 +212,7 @@ The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **
|
|
| 212 |
| Desktop + discrete GPU | i7-8700K (6C/12T) | **NVIDIA RTX 5070 Ti** 16 GB | 32 GB | 9.7 | **0.9** ⭐ | 1.5 |
|
| 213 |
| Laptop (Arrow Lake-H) | Core Ultra 7 255H (16C/16T) | Intel **Arc 140T** iGPU | 48 GB | **4.6** | 8.6 | **1.6** ⭐ |
|
| 214 |
| Laptop (Tiger Lake) | i7-1165G7 (4C/8T) | Intel Iris Xe 96 EU iGPU | 16 GB | 13.9 | **9.6** ⭐ | ✗ OOM at init |
|
| 215 |
-
| Laptop (Raptor Lake-U) | i5-1335U (10C/12T, 15 W) | Intel Iris Xe 80 EU iGPU | 8 GB | **≈15** ⭐ | ✗ OOM ~3934 MB¹ | ✗ device-hung |
|
| 216 |
|
| 217 |
⭐ = fastest working backend on that machine. ✗ = execution provider failed (footnotes ¹–³ below give the byte-level cause). Numbers here are the steady-state median across sizes; the **per-size mean ± std** is in the Reference anchors tables below. Add **pre+post** for end-to-end total: ≈0.05 s at 512²/1080p, ≈0.5 s at 8K.
|
| 218 |
|
|
@@ -220,7 +220,7 @@ The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **
|
|
| 220 |
|
| 221 |
### Reference anchors — match your own machine
|
| 222 |
|
| 223 |
-
Every **hardware × EP combination measured**, broken out by input size. Cells are **mean ± std** of the per-image **inference** time in seconds (the warm-up run is dropped, so n ≈ 14; n = 10 where noted). ✗ = the EP failed — see footnotes. Because inference is ~size-independent (static 1024² input) a row's four cells are nearly equal; the size axis mainly exposes **where memory runs out** (e.g. the 8 GB laptop clears 8K but not the smaller sizes). fp16 TFLOPS are a published **proxy** for ranking GPUs (the model runs fp32, so real rates are lower); discrete-GPU bandwidth is on-card VRAM, iGPU bandwidth is the shared-RAM `memcpy` rate measured. `″` = same as the row above; the Arc 140T additionally has XMX matrix engines (74 INT8 TOPS) that its fp32 path does not use.
|
| 224 |
|
| 225 |
**GPU backends** (anchor = fp16 throughput + memory bandwidth):
|
| 226 |
|
|
@@ -228,34 +228,34 @@ Every **hardware × EP combination measured**, broken out by input size. Cells a
|
|
| 228 |
|---|---|--:|--:|--:|--:|--:|--:|
|
| 229 |
| RTX 5070 Ti (Blackwell), discrete 16 GB | WebGPU | 88 TFLOPS | 896 GB/s | 0.95 ± 0.07 | 0.85 ± 0.06 | 0.85 ± 0.07 | 0.89 ± 0.08 |
|
| 230 |
| RTX 5070 Ti | DirectML | ″ | ″ | 1.50 ± 0.03 | 1.50 ± 0.06 | 1.48 ± 0.04 | 1.50 ± 0.04 |
|
| 231 |
-
| Arc 140T iGPU (Xe-LPG, 8 Xe @2.25 GHz) | DirectML | ~9 TFLOPS | ~39 GB/s sh. | 1.58 ± 0.02 | 1.57 ± 0.00 | 1.57 ± 0.00 | 1.57 ± 0.00 |
|
| 232 |
| Arc 140T | WebGPU | ″ | ″ | 8.61 ± 0.05 | 8.64 ± 0.08 | 8.64 ± 0.09 | 8.65 ± 0.07 |
|
| 233 |
-
| Iris Xe 96 EU iGPU (Tiger Lake) | WebGPU | 3.4 TFLOPS | ~31 GB/s sh. | 9.79 ± 0.40 | 9.52 ± 0.34 | 9.46 ± 0.25 | 9.77 ± 0.46 |
|
| 234 |
| Iris Xe 96 EU | DirectML | ″ | ″ | ✗² | ✗² | ✗² | ✗² |
|
| 235 |
-
| Iris Xe 80 EU iGPU (Raptor Lake-U) | WebGPU | ~3 TFLOPS | ~29 GB/s sh. | 33.4 ± 2.8¹ | ✗¹ | ✗¹ | ✗¹ |
|
| 236 |
| Iris Xe 80 EU | DirectML | ″ | ″ | ✗³ | ✗³ | ✗³ | ✗³ |
|
| 237 |
|
| 238 |
-
¹ **WebGPU / Iris Xe 80 EU (8 GB RAM):** peaked at **~3934 MB — essentially the entire ~3940 MB shared budget** (≈50% of 8 GB RAM). Only 8K ran (11 of 15 runs, ~33 s mean) before it OOM'd; every 4K / 1080p / 512 run failed. WebGPU is **unusable at 8 GB RAM**. <br>
|
| 239 |
-
² **DirectML / Iris Xe 96 EU (16 GB RAM):** out of memory (`8007000E`) **during session creation** (~12.7 s in) — never ran. DirectML's footprint is **~16–19 GB** (peak where it ran) vs only **~8032 MB** shared iGPU budget (≈50% of 16 GB RAM). <br>
|
| 240 |
-
³ **DirectML / Iris Xe 80 EU (8 GB RAM):** init failed with a GPU **device-hung** error (`887A0007`, "GPU will not respond to more commands") — not strictly OOM, likely a cascade from the preceding WebGPU crash on the same ~3940 MB shared budget.
|
| 241 |
|
| 242 |
**CPU backend** (anchor = cores × clock × AVX width + RAM bandwidth; EP = CPU on every row):
|
| 243 |
|
| 244 |
| CPU (cores/threads, arch) | RAM BW | 8K (s) | 4K (s) | 1080p (s) | 512² (s) |
|
| 245 |
|---|--:|--:|--:|--:|--:|
|
| 246 |
-
| Core Ultra 7 255H (16C/16T, Arrow Lake-H, AVX2) | ~39 GB/s | 4.57 ± 0.08 | 4.61 ± 0.05 | 4.63 ± 0.12 | 4.65 ± 0.17 |
|
| 247 |
-
| i7-8700K (6C/12T, Coffee Lake, AVX2) | ~25 GB/s | 9.75 ± 0.24 | 9.69 ± 0.10 | 9.67 ± 0.11 | 9.67 ± 0.13 |
|
| 248 |
-
| i7-1165G7 (4C/8T, Tiger Lake, AVX2/512) | ~31 GB/s | 14.6 ± 1.5 | 13.9 ± 0.5 | 13.8 ± 0.3 | 13.9 ± 0.2 |
|
| 249 |
-
| i5-1335U (10C/12T, 15 W, Raptor Lake-U) | ~29 GB/s | 14.3 ± 2.1 | 14.9 ± 1.6 | 15.7 ± 2.0 | 14.3 ± 1.8 |
|
| 250 |
|
| 251 |
(The CPU EP was also run on the RTX 5070 Ti box — that's the i7-8700K row — since the CPU EP ignores the GPU. The 1335U's larger ± is real run-to-run jitter on a 15 W thermally-throttled laptop.)
|
| 252 |
|
| 253 |
**How to read it / rules of thumb:**
|
| 254 |
|
| 255 |
-
- **Choose the EP, not just the chip.** Discrete NVIDIA/AMD → **WebGPU** (fastest). Intel **Arc** iGPU → **DirectML** (its WebGPU path is ~5× slower here). Older Intel **Iris Xe** → **WebGPU** if you have ≥16 GB RAM, else **CPU**. No usable GPU → **CPU** (scales with cores × clock × AVX width; the 16-core 255H is ~3× the 4-core 1165G7).
|
| 256 |
-
- **RAM is the gate for iGPUs.** The fp32 model is ~805 MB on disk but a GPU session peaks at **8–20 GB** (VRAM on discrete cards; *shared system RAM* on iGPUs — WebGPU peaked ~8 GB, DirectML ~15–19 GB in our runs). So: iGPU WebGPU needs **≥16 GB system RAM**; iGPU DirectML wants **≥32 GB**; **8 GB machines can only use the CPU**. Discrete GPUs need **≥8 GB VRAM** free.
|
| 257 |
- **DirectML is hit-or-miss.** It gave the best iGPU time (Arc 140T, 1.6 s) but failed to initialize on three of four machines (driver/VRAM dependent). Always have the CPU EP as a fallback.
|
| 258 |
-
- **Predict GPU time** ≈ scale the RTX 5070 Ti's 0.9 s by its fp16 proxy vs yours, then add ~30–50 % headroom because the model is fp32 and won't fully use tensor cores. (e.g. a ~40 TFLOPS-fp16 card → expect ~2 s; a ~9 TFLOPS iGPU via a *good* runtime → ~1.5–2 s, which matches the Arc 140T on DML.)
|
| 259 |
|
| 260 |
## Notes
|
| 261 |
|
|
|
|
| 24 |
FP32 ONNX export of [**finegrain/finegrain-box-segmenter**](https://huggingface.co/finegrain/finegrain-box-segmenter) (an MVANet high-resolution background remover), ready to run in the browser, Node.js and Python with [ONNX Runtime](https://onnxruntime.ai/) (CPU, WebGPU (recommended), DirectML).
|
| 25 |
|
| 26 |
- **Base model:** `finegrain/finegrain-box-segmenter` (MIT) — MVANet, SafeTensors, [arXiv:2404.07445](https://arxiv.org/abs/2404.07445)
|
| 27 |
+
- **Architecture:** MVANet with a **Swin-B** backbone, **\~94.6 M parameters**; static 1024×1024
|
| 28 |
input, batch fixed at 1.
|
| 29 |
- **What this repo adds:** the same network exported to ONNX (full-precision floating-point **fp32**), so it runs
|
| 30 |
without PyTorch/refiners.
|
| 31 |
+
- **Why fp32:** maximum compatibility with zero precision risk — the fp32 graph runs on CPU, WebGPU and DirectML, and stays **bit-identical** to the original release; at \~94.6 M parameters it is still cheap enough to run on a local machine. (An fp16 build (\~403 MB) exists and its masks matched fp32 in testing, but it is not published, to keep the release simple — open a discussion if you need it.) Regarding why this release uses the "quantized" tag: it's a fairly common convention for ONNX / format derivatives (same as e.g. `onnx-community/BiRefNet-ONNX`), so I decided to go this route for tagging.
|
| 32 |
- **Output is bit-identical to the PyTorch reference** on CPU (mask MAE `0.000 / 255`; random-input logits `max|Δ| = 1.5e-5`).
|
| 33 |
|
| 34 |
## Examples
|
|
|
|
| 74 |
|
| 75 |
| File | Precision | Size | Notes |
|
| 76 |
|------|-----------|-----:|-------|
|
| 77 |
+
| `onnx/model.onnx` | fp32 | \~805 MB | full precision; CPU + WebGPU; bit-identical parity with PyTorch |
|
| 78 |
|
| 79 |
+
ONNX operator set version 17, \~94.6 M parameters, SHA256(onnx/model.onnx) = `d3f60fc778f2b14435a8df3d61511c293cec3a88c670ad8ecac5035c6b9730d5`
|
| 80 |
|
| 81 |
+
> Why 805 MB for a 94.6 M-param model: \~378 MB is fp32 weights; the other \~425 MB is constant-folded shifted-window (Swin) attention masks / position embeddings baked in at export (`do_constant_folding=True`). Note that disabling the folding will make the file *larger*, not smaller.
|
| 82 |
|
| 83 |
## I/O contract
|
| 84 |
|
|
|
|
| 179 |
|
| 180 |
## Usage — Python (onnxruntime)
|
| 181 |
|
| 182 |
+
Verified against this exact export (`pip install onnxruntime pillow numpy huggingface_hub`); masks match the Node reference pipeline to \~1/255 mean difference:
|
| 183 |
|
| 184 |
```python
|
| 185 |
from huggingface_hub import hf_hub_download
|
|
|
|
| 203 |
|
| 204 |
## Performance across environments
|
| 205 |
|
| 206 |
+
The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **batch-5**, so it is **compute-bound and the inference time is \~constant regardless of input image size** — the source resolution only changes the negligible host-side resize (`pre` + `post`, ≈30–500 ms for 1080p→8K). The numbers below are therefore "**per image**" for any resolution. All EPs run the **same fp32 model** (no fp16/NPU path). Measured with `onnxruntime-node` 1.26, **15 runs/cell, median**.
|
| 207 |
|
| 208 |
### Measured — median inference time, seconds per image (lower is better)
|
| 209 |
|
|
|
|
| 212 |
| Desktop + discrete GPU | i7-8700K (6C/12T) | **NVIDIA RTX 5070 Ti** 16 GB | 32 GB | 9.7 | **0.9** ⭐ | 1.5 |
|
| 213 |
| Laptop (Arrow Lake-H) | Core Ultra 7 255H (16C/16T) | Intel **Arc 140T** iGPU | 48 GB | **4.6** | 8.6 | **1.6** ⭐ |
|
| 214 |
| Laptop (Tiger Lake) | i7-1165G7 (4C/8T) | Intel Iris Xe 96 EU iGPU | 16 GB | 13.9 | **9.6** ⭐ | ✗ OOM at init |
|
| 215 |
+
| Laptop (Raptor Lake-U) | i5-1335U (10C/12T, 15 W) | Intel Iris Xe 80 EU iGPU | 8 GB | **≈15** ⭐ | ✗ OOM \~3934 MB¹ | ✗ device-hung |
|
| 216 |
|
| 217 |
⭐ = fastest working backend on that machine. ✗ = execution provider failed (footnotes ¹–³ below give the byte-level cause). Numbers here are the steady-state median across sizes; the **per-size mean ± std** is in the Reference anchors tables below. Add **pre+post** for end-to-end total: ≈0.05 s at 512²/1080p, ≈0.5 s at 8K.
|
| 218 |
|
|
|
|
| 220 |
|
| 221 |
### Reference anchors — match your own machine
|
| 222 |
|
| 223 |
+
Every **hardware × EP combination measured**, broken out by input size. Cells are **mean ± std** of the per-image **inference** time in seconds (the warm-up run is dropped, so n ≈ 14; n = 10 where noted). ✗ = the EP failed — see footnotes. Because inference is \~size-independent (static 1024² input) a row's four cells are nearly equal; the size axis mainly exposes **where memory runs out** (e.g. the 8 GB laptop clears 8K but not the smaller sizes). fp16 TFLOPS are a published **proxy** for ranking GPUs (the model runs fp32, so real rates are lower); discrete-GPU bandwidth is on-card VRAM, iGPU bandwidth is the shared-RAM `memcpy` rate measured. `″` = same as the row above; the Arc 140T additionally has XMX matrix engines (74 INT8 TOPS) that its fp32 path does not use.
|
| 224 |
|
| 225 |
**GPU backends** (anchor = fp16 throughput + memory bandwidth):
|
| 226 |
|
|
|
|
| 228 |
|---|---|--:|--:|--:|--:|--:|--:|
|
| 229 |
| RTX 5070 Ti (Blackwell), discrete 16 GB | WebGPU | 88 TFLOPS | 896 GB/s | 0.95 ± 0.07 | 0.85 ± 0.06 | 0.85 ± 0.07 | 0.89 ± 0.08 |
|
| 230 |
| RTX 5070 Ti | DirectML | ″ | ″ | 1.50 ± 0.03 | 1.50 ± 0.06 | 1.48 ± 0.04 | 1.50 ± 0.04 |
|
| 231 |
+
| Arc 140T iGPU (Xe-LPG, 8 Xe @2.25 GHz) | DirectML | \~9 TFLOPS | \~39 GB/s sh. | 1.58 ± 0.02 | 1.57 ± 0.00 | 1.57 ± 0.00 | 1.57 ± 0.00 |
|
| 232 |
| Arc 140T | WebGPU | ″ | ″ | 8.61 ± 0.05 | 8.64 ± 0.08 | 8.64 ± 0.09 | 8.65 ± 0.07 |
|
| 233 |
+
| Iris Xe 96 EU iGPU (Tiger Lake) | WebGPU | 3.4 TFLOPS | \~31 GB/s sh. | 9.79 ± 0.40 | 9.52 ± 0.34 | 9.46 ± 0.25 | 9.77 ± 0.46 |
|
| 234 |
| Iris Xe 96 EU | DirectML | ″ | ″ | ✗² | ✗² | ✗² | ✗² |
|
| 235 |
+
| Iris Xe 80 EU iGPU (Raptor Lake-U) | WebGPU | \~3 TFLOPS | \~29 GB/s sh. | 33.4 ± 2.8¹ | ✗¹ | ✗¹ | ✗¹ |
|
| 236 |
| Iris Xe 80 EU | DirectML | ″ | ″ | ✗³ | ✗³ | ✗³ | ✗³ |
|
| 237 |
|
| 238 |
+
¹ **WebGPU / Iris Xe 80 EU (8 GB RAM):** peaked at **\~3934 MB — essentially the entire \~3940 MB shared budget** (≈50% of 8 GB RAM). Only 8K ran (11 of 15 runs, \~33 s mean) before it OOM'd; every 4K / 1080p / 512 run failed. WebGPU is **unusable at 8 GB RAM**. <br>
|
| 239 |
+
² **DirectML / Iris Xe 96 EU (16 GB RAM):** out of memory (`8007000E`) **during session creation** (\~12.7 s in) — never ran. DirectML's footprint is **\~16–19 GB** (peak where it ran) vs only **\~8032 MB** shared iGPU budget (≈50% of 16 GB RAM). <br>
|
| 240 |
+
³ **DirectML / Iris Xe 80 EU (8 GB RAM):** init failed with a GPU **device-hung** error (`887A0007`, "GPU will not respond to more commands") — not strictly OOM, likely a cascade from the preceding WebGPU crash on the same \~3940 MB shared budget.
|
| 241 |
|
| 242 |
**CPU backend** (anchor = cores × clock × AVX width + RAM bandwidth; EP = CPU on every row):
|
| 243 |
|
| 244 |
| CPU (cores/threads, arch) | RAM BW | 8K (s) | 4K (s) | 1080p (s) | 512² (s) |
|
| 245 |
|---|--:|--:|--:|--:|--:|
|
| 246 |
+
| Core Ultra 7 255H (16C/16T, Arrow Lake-H, AVX2) | \~39 GB/s | 4.57 ± 0.08 | 4.61 ± 0.05 | 4.63 ± 0.12 | 4.65 ± 0.17 |
|
| 247 |
+
| i7-8700K (6C/12T, Coffee Lake, AVX2) | \~25 GB/s | 9.75 ± 0.24 | 9.69 ± 0.10 | 9.67 ± 0.11 | 9.67 ± 0.13 |
|
| 248 |
+
| i7-1165G7 (4C/8T, Tiger Lake, AVX2/512) | \~31 GB/s | 14.6 ± 1.5 | 13.9 ± 0.5 | 13.8 ± 0.3 | 13.9 ± 0.2 |
|
| 249 |
+
| i5-1335U (10C/12T, 15 W, Raptor Lake-U) | \~29 GB/s | 14.3 ± 2.1 | 14.9 ± 1.6 | 15.7 ± 2.0 | 14.3 ± 1.8 |
|
| 250 |
|
| 251 |
(The CPU EP was also run on the RTX 5070 Ti box — that's the i7-8700K row — since the CPU EP ignores the GPU. The 1335U's larger ± is real run-to-run jitter on a 15 W thermally-throttled laptop.)
|
| 252 |
|
| 253 |
**How to read it / rules of thumb:**
|
| 254 |
|
| 255 |
+
- **Choose the EP, not just the chip.** Discrete NVIDIA/AMD → **WebGPU** (fastest). Intel **Arc** iGPU → **DirectML** (its WebGPU path is \~5× slower here). Older Intel **Iris Xe** → **WebGPU** if you have ≥16 GB RAM, else **CPU**. No usable GPU → **CPU** (scales with cores × clock × AVX width; the 16-core 255H is \~3× the 4-core 1165G7).
|
| 256 |
+
- **RAM is the gate for iGPUs.** The fp32 model is \~805 MB on disk but a GPU session peaks at **8–20 GB** (VRAM on discrete cards; *shared system RAM* on iGPUs — WebGPU peaked \~8 GB, DirectML \~15–19 GB in our runs). So: iGPU WebGPU needs **≥16 GB system RAM**; iGPU DirectML wants **≥32 GB**; **8 GB machines can only use the CPU**. Discrete GPUs need **≥8 GB VRAM** free.
|
| 257 |
- **DirectML is hit-or-miss.** It gave the best iGPU time (Arc 140T, 1.6 s) but failed to initialize on three of four machines (driver/VRAM dependent). Always have the CPU EP as a fallback.
|
| 258 |
+
- **Predict GPU time** ≈ scale the RTX 5070 Ti's 0.9 s by its fp16 proxy vs yours, then add \~30–50 % headroom because the model is fp32 and won't fully use tensor cores. (e.g. a \~40 TFLOPS-fp16 card → expect \~2 s; a \~9 TFLOPS iGPU via a *good* runtime → \~1.5–2 s, which matches the Arc 140T on DML.)
|
| 259 |
|
| 260 |
## Notes
|
| 261 |
|