MarcinEU commited on
Commit
7cee2ee
·
1 Parent(s): 24c7465

card: escape tildes to prevent strikethrough rendering

Browse files
Files changed (1) hide show
  1. README.md +22 -22
README.md CHANGED
@@ -24,11 +24,11 @@ tags:
24
  FP32 ONNX export of [**finegrain/finegrain-box-segmenter**](https://huggingface.co/finegrain/finegrain-box-segmenter) (an MVANet high-resolution background remover), ready to run in the browser, Node.js and Python with [ONNX Runtime](https://onnxruntime.ai/) (CPU, WebGPU (recommended), DirectML).
25
 
26
  - **Base model:** `finegrain/finegrain-box-segmenter` (MIT) — MVANet, SafeTensors, [arXiv:2404.07445](https://arxiv.org/abs/2404.07445)
27
- - **Architecture:** MVANet with a **Swin-B** backbone, **~94.6 M parameters**; static 1024×1024
28
  input, batch fixed at 1.
29
  - **What this repo adds:** the same network exported to ONNX (full-precision floating-point **fp32**), so it runs
30
  without PyTorch/refiners.
31
- - **Why fp32:** maximum compatibility with zero precision risk — the fp32 graph runs on CPU, WebGPU and DirectML, and stays **bit-identical** to the original release; at ~94.6 M parameters it is still cheap enough to run on a local machine. (An fp16 build (\~403 MB) exists and its masks matched fp32 in testing, but it is not published, to keep the release simple — open a discussion if you need it.) Regarding why this release uses the "quantized" tag: it's a fairly common convention for ONNX / format derivatives (same as e.g. `onnx-community/BiRefNet-ONNX`), so I decided to go this route for tagging.
32
  - **Output is bit-identical to the PyTorch reference** on CPU (mask MAE `0.000 / 255`; random-input logits `max|Δ| = 1.5e-5`).
33
 
34
  ## Examples
@@ -74,11 +74,11 @@ The short version: **finegrain-box-segmenter is strongest on soft, semi-transpar
74
 
75
  | File | Precision | Size | Notes |
76
  |------|-----------|-----:|-------|
77
- | `onnx/model.onnx` | fp32 | ~805 MB | full precision; CPU + WebGPU; bit-identical parity with PyTorch |
78
 
79
- ONNX operator set version 17, ~94.6 M parameters, SHA256(onnx/model.onnx) = `d3f60fc778f2b14435a8df3d61511c293cec3a88c670ad8ecac5035c6b9730d5`
80
 
81
- > Why 805 MB for a 94.6 M-param model: ~378 MB is fp32 weights; the other ~425 MB is constant-folded shifted-window (Swin) attention masks / position embeddings baked in at export (`do_constant_folding=True`). Note that disabling the folding will make the file *larger*, not smaller.
82
 
83
  ## I/O contract
84
 
@@ -179,7 +179,7 @@ It resizes the whole image to 1024², normalizes, runs the session, applies sigm
179
 
180
  ## Usage — Python (onnxruntime)
181
 
182
- Verified against this exact export (`pip install onnxruntime pillow numpy huggingface_hub`); masks match the Node reference pipeline to ~1/255 mean difference:
183
 
184
  ```python
185
  from huggingface_hub import hf_hub_download
@@ -203,7 +203,7 @@ cutout.save("cutout.png"); mask.save("mask.png")
203
 
204
  ## Performance across environments
205
 
206
- The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **batch-5**, so it is **compute-bound and the inference time is ~constant regardless of input image size** — the source resolution only changes the negligible host-side resize (`pre` + `post`, ≈30–500 ms for 1080p→8K). The numbers below are therefore "**per image**" for any resolution. All EPs run the **same fp32 model** (no fp16/NPU path). Measured with `onnxruntime-node` 1.26, **15 runs/cell, median**.
207
 
208
  ### Measured — median inference time, seconds per image (lower is better)
209
 
@@ -212,7 +212,7 @@ The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **
212
  | Desktop + discrete GPU | i7-8700K (6C/12T) | **NVIDIA RTX 5070 Ti** 16 GB | 32 GB | 9.7 | **0.9** ⭐ | 1.5 |
213
  | Laptop (Arrow Lake-H) | Core Ultra 7 255H (16C/16T) | Intel **Arc 140T** iGPU | 48 GB | **4.6** | 8.6 | **1.6** ⭐ |
214
  | Laptop (Tiger Lake) | i7-1165G7 (4C/8T) | Intel Iris Xe 96 EU iGPU | 16 GB | 13.9 | **9.6** ⭐ | ✗ OOM at init |
215
- | Laptop (Raptor Lake-U) | i5-1335U (10C/12T, 15 W) | Intel Iris Xe 80 EU iGPU | 8 GB | **≈15** ⭐ | ✗ OOM ~3934 MB¹ | ✗ device-hung |
216
 
217
  ⭐ = fastest working backend on that machine. ✗ = execution provider failed (footnotes ¹–³ below give the byte-level cause). Numbers here are the steady-state median across sizes; the **per-size mean ± std** is in the Reference anchors tables below. Add **pre+post** for end-to-end total: ≈0.05 s at 512²/1080p, ≈0.5 s at 8K.
218
 
@@ -220,7 +220,7 @@ The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **
220
 
221
  ### Reference anchors — match your own machine
222
 
223
- Every **hardware × EP combination measured**, broken out by input size. Cells are **mean ± std** of the per-image **inference** time in seconds (the warm-up run is dropped, so n ≈ 14; n = 10 where noted). ✗ = the EP failed — see footnotes. Because inference is ~size-independent (static 1024² input) a row's four cells are nearly equal; the size axis mainly exposes **where memory runs out** (e.g. the 8 GB laptop clears 8K but not the smaller sizes). fp16 TFLOPS are a published **proxy** for ranking GPUs (the model runs fp32, so real rates are lower); discrete-GPU bandwidth is on-card VRAM, iGPU bandwidth is the shared-RAM `memcpy` rate measured. `″` = same as the row above; the Arc 140T additionally has XMX matrix engines (74 INT8 TOPS) that its fp32 path does not use.
224
 
225
  **GPU backends** (anchor = fp16 throughput + memory bandwidth):
226
 
@@ -228,34 +228,34 @@ Every **hardware × EP combination measured**, broken out by input size. Cells a
228
  |---|---|--:|--:|--:|--:|--:|--:|
229
  | RTX 5070 Ti (Blackwell), discrete 16 GB | WebGPU | 88 TFLOPS | 896 GB/s | 0.95 ± 0.07 | 0.85 ± 0.06 | 0.85 ± 0.07 | 0.89 ± 0.08 |
230
  | RTX 5070 Ti | DirectML | ″ | ″ | 1.50 ± 0.03 | 1.50 ± 0.06 | 1.48 ± 0.04 | 1.50 ± 0.04 |
231
- | Arc 140T iGPU (Xe-LPG, 8 Xe @2.25 GHz) | DirectML | ~9 TFLOPS | ~39 GB/s sh. | 1.58 ± 0.02 | 1.57 ± 0.00 | 1.57 ± 0.00 | 1.57 ± 0.00 |
232
  | Arc 140T | WebGPU | ″ | ″ | 8.61 ± 0.05 | 8.64 ± 0.08 | 8.64 ± 0.09 | 8.65 ± 0.07 |
233
- | Iris Xe 96 EU iGPU (Tiger Lake) | WebGPU | 3.4 TFLOPS | ~31 GB/s sh. | 9.79 ± 0.40 | 9.52 ± 0.34 | 9.46 ± 0.25 | 9.77 ± 0.46 |
234
  | Iris Xe 96 EU | DirectML | ″ | ″ | ✗² | ✗² | ✗² | ✗² |
235
- | Iris Xe 80 EU iGPU (Raptor Lake-U) | WebGPU | ~3 TFLOPS | ~29 GB/s sh. | 33.4 ± 2.8¹ | ✗¹ | ✗¹ | ✗¹ |
236
  | Iris Xe 80 EU | DirectML | ″ | ″ | ✗³ | ✗³ | ✗³ | ✗³ |
237
 
238
- ¹ **WebGPU / Iris Xe 80 EU (8 GB RAM):** peaked at **~3934 MB — essentially the entire ~3940 MB shared budget** (≈50% of 8 GB RAM). Only 8K ran (11 of 15 runs, ~33 s mean) before it OOM'd; every 4K / 1080p / 512 run failed. WebGPU is **unusable at 8 GB RAM**. <br>
239
- ² **DirectML / Iris Xe 96 EU (16 GB RAM):** out of memory (`8007000E`) **during session creation** (~12.7 s in) — never ran. DirectML's footprint is **~16–19 GB** (peak where it ran) vs only **~8032 MB** shared iGPU budget (≈50% of 16 GB RAM). <br>
240
- ³ **DirectML / Iris Xe 80 EU (8 GB RAM):** init failed with a GPU **device-hung** error (`887A0007`, "GPU will not respond to more commands") — not strictly OOM, likely a cascade from the preceding WebGPU crash on the same ~3940 MB shared budget.
241
 
242
  **CPU backend** (anchor = cores × clock × AVX width + RAM bandwidth; EP = CPU on every row):
243
 
244
  | CPU (cores/threads, arch) | RAM BW | 8K (s) | 4K (s) | 1080p (s) | 512² (s) |
245
  |---|--:|--:|--:|--:|--:|
246
- | Core Ultra 7 255H (16C/16T, Arrow Lake-H, AVX2) | ~39 GB/s | 4.57 ± 0.08 | 4.61 ± 0.05 | 4.63 ± 0.12 | 4.65 ± 0.17 |
247
- | i7-8700K (6C/12T, Coffee Lake, AVX2) | ~25 GB/s | 9.75 ± 0.24 | 9.69 ± 0.10 | 9.67 ± 0.11 | 9.67 ± 0.13 |
248
- | i7-1165G7 (4C/8T, Tiger Lake, AVX2/512) | ~31 GB/s | 14.6 ± 1.5 | 13.9 ± 0.5 | 13.8 ± 0.3 | 13.9 ± 0.2 |
249
- | i5-1335U (10C/12T, 15 W, Raptor Lake-U) | ~29 GB/s | 14.3 ± 2.1 | 14.9 ± 1.6 | 15.7 ± 2.0 | 14.3 ± 1.8 |
250
 
251
  (The CPU EP was also run on the RTX 5070 Ti box — that's the i7-8700K row — since the CPU EP ignores the GPU. The 1335U's larger ± is real run-to-run jitter on a 15 W thermally-throttled laptop.)
252
 
253
  **How to read it / rules of thumb:**
254
 
255
- - **Choose the EP, not just the chip.** Discrete NVIDIA/AMD → **WebGPU** (fastest). Intel **Arc** iGPU → **DirectML** (its WebGPU path is ~5× slower here). Older Intel **Iris Xe** → **WebGPU** if you have ≥16 GB RAM, else **CPU**. No usable GPU → **CPU** (scales with cores × clock × AVX width; the 16-core 255H is ~3× the 4-core 1165G7).
256
- - **RAM is the gate for iGPUs.** The fp32 model is ~805 MB on disk but a GPU session peaks at **8–20 GB** (VRAM on discrete cards; *shared system RAM* on iGPUs — WebGPU peaked ~8 GB, DirectML ~15–19 GB in our runs). So: iGPU WebGPU needs **≥16 GB system RAM**; iGPU DirectML wants **≥32 GB**; **8 GB machines can only use the CPU**. Discrete GPUs need **≥8 GB VRAM** free.
257
  - **DirectML is hit-or-miss.** It gave the best iGPU time (Arc 140T, 1.6 s) but failed to initialize on three of four machines (driver/VRAM dependent). Always have the CPU EP as a fallback.
258
- - **Predict GPU time** ≈ scale the RTX 5070 Ti's 0.9 s by its fp16 proxy vs yours, then add ~30–50 % headroom because the model is fp32 and won't fully use tensor cores. (e.g. a ~40 TFLOPS-fp16 card → expect ~2 s; a ~9 TFLOPS iGPU via a *good* runtime → ~1.5–2 s, which matches the Arc 140T on DML.)
259
 
260
  ## Notes
261
 
 
24
  FP32 ONNX export of [**finegrain/finegrain-box-segmenter**](https://huggingface.co/finegrain/finegrain-box-segmenter) (an MVANet high-resolution background remover), ready to run in the browser, Node.js and Python with [ONNX Runtime](https://onnxruntime.ai/) (CPU, WebGPU (recommended), DirectML).
25
 
26
  - **Base model:** `finegrain/finegrain-box-segmenter` (MIT) — MVANet, SafeTensors, [arXiv:2404.07445](https://arxiv.org/abs/2404.07445)
27
+ - **Architecture:** MVANet with a **Swin-B** backbone, **\~94.6 M parameters**; static 1024×1024
28
  input, batch fixed at 1.
29
  - **What this repo adds:** the same network exported to ONNX (full-precision floating-point **fp32**), so it runs
30
  without PyTorch/refiners.
31
+ - **Why fp32:** maximum compatibility with zero precision risk — the fp32 graph runs on CPU, WebGPU and DirectML, and stays **bit-identical** to the original release; at \~94.6 M parameters it is still cheap enough to run on a local machine. (An fp16 build (\~403 MB) exists and its masks matched fp32 in testing, but it is not published, to keep the release simple — open a discussion if you need it.) Regarding why this release uses the "quantized" tag: it's a fairly common convention for ONNX / format derivatives (same as e.g. `onnx-community/BiRefNet-ONNX`), so I decided to go this route for tagging.
32
  - **Output is bit-identical to the PyTorch reference** on CPU (mask MAE `0.000 / 255`; random-input logits `max|Δ| = 1.5e-5`).
33
 
34
  ## Examples
 
74
 
75
  | File | Precision | Size | Notes |
76
  |------|-----------|-----:|-------|
77
+ | `onnx/model.onnx` | fp32 | \~805 MB | full precision; CPU + WebGPU; bit-identical parity with PyTorch |
78
 
79
+ ONNX operator set version 17, \~94.6 M parameters, SHA256(onnx/model.onnx) = `d3f60fc778f2b14435a8df3d61511c293cec3a88c670ad8ecac5035c6b9730d5`
80
 
81
+ > Why 805 MB for a 94.6 M-param model: \~378 MB is fp32 weights; the other \~425 MB is constant-folded shifted-window (Swin) attention masks / position embeddings baked in at export (`do_constant_folding=True`). Note that disabling the folding will make the file *larger*, not smaller.
82
 
83
  ## I/O contract
84
 
 
179
 
180
  ## Usage — Python (onnxruntime)
181
 
182
+ Verified against this exact export (`pip install onnxruntime pillow numpy huggingface_hub`); masks match the Node reference pipeline to \~1/255 mean difference:
183
 
184
  ```python
185
  from huggingface_hub import hf_hub_download
 
203
 
204
  ## Performance across environments
205
 
206
+ The graph is a **static 1024×1024 fp32** Swin-B (MVANet) running at internal **batch-5**, so it is **compute-bound and the inference time is \~constant regardless of input image size** — the source resolution only changes the negligible host-side resize (`pre` + `post`, ≈30–500 ms for 1080p→8K). The numbers below are therefore "**per image**" for any resolution. All EPs run the **same fp32 model** (no fp16/NPU path). Measured with `onnxruntime-node` 1.26, **15 runs/cell, median**.
207
 
208
  ### Measured — median inference time, seconds per image (lower is better)
209
 
 
212
  | Desktop + discrete GPU | i7-8700K (6C/12T) | **NVIDIA RTX 5070 Ti** 16 GB | 32 GB | 9.7 | **0.9** ⭐ | 1.5 |
213
  | Laptop (Arrow Lake-H) | Core Ultra 7 255H (16C/16T) | Intel **Arc 140T** iGPU | 48 GB | **4.6** | 8.6 | **1.6** ⭐ |
214
  | Laptop (Tiger Lake) | i7-1165G7 (4C/8T) | Intel Iris Xe 96 EU iGPU | 16 GB | 13.9 | **9.6** ⭐ | ✗ OOM at init |
215
+ | Laptop (Raptor Lake-U) | i5-1335U (10C/12T, 15 W) | Intel Iris Xe 80 EU iGPU | 8 GB | **≈15** ⭐ | ✗ OOM \~3934 MB¹ | ✗ device-hung |
216
 
217
  ⭐ = fastest working backend on that machine. ✗ = execution provider failed (footnotes ¹–³ below give the byte-level cause). Numbers here are the steady-state median across sizes; the **per-size mean ± std** is in the Reference anchors tables below. Add **pre+post** for end-to-end total: ≈0.05 s at 512²/1080p, ≈0.5 s at 8K.
218
 
 
220
 
221
  ### Reference anchors — match your own machine
222
 
223
+ Every **hardware × EP combination measured**, broken out by input size. Cells are **mean ± std** of the per-image **inference** time in seconds (the warm-up run is dropped, so n ≈ 14; n = 10 where noted). ✗ = the EP failed — see footnotes. Because inference is \~size-independent (static 1024² input) a row's four cells are nearly equal; the size axis mainly exposes **where memory runs out** (e.g. the 8 GB laptop clears 8K but not the smaller sizes). fp16 TFLOPS are a published **proxy** for ranking GPUs (the model runs fp32, so real rates are lower); discrete-GPU bandwidth is on-card VRAM, iGPU bandwidth is the shared-RAM `memcpy` rate measured. `″` = same as the row above; the Arc 140T additionally has XMX matrix engines (74 INT8 TOPS) that its fp32 path does not use.
224
 
225
  **GPU backends** (anchor = fp16 throughput + memory bandwidth):
226
 
 
228
  |---|---|--:|--:|--:|--:|--:|--:|
229
  | RTX 5070 Ti (Blackwell), discrete 16 GB | WebGPU | 88 TFLOPS | 896 GB/s | 0.95 ± 0.07 | 0.85 ± 0.06 | 0.85 ± 0.07 | 0.89 ± 0.08 |
230
  | RTX 5070 Ti | DirectML | ″ | ″ | 1.50 ± 0.03 | 1.50 ± 0.06 | 1.48 ± 0.04 | 1.50 ± 0.04 |
231
+ | Arc 140T iGPU (Xe-LPG, 8 Xe @2.25 GHz) | DirectML | \~9 TFLOPS | \~39 GB/s sh. | 1.58 ± 0.02 | 1.57 ± 0.00 | 1.57 ± 0.00 | 1.57 ± 0.00 |
232
  | Arc 140T | WebGPU | ″ | ″ | 8.61 ± 0.05 | 8.64 ± 0.08 | 8.64 ± 0.09 | 8.65 ± 0.07 |
233
+ | Iris Xe 96 EU iGPU (Tiger Lake) | WebGPU | 3.4 TFLOPS | \~31 GB/s sh. | 9.79 ± 0.40 | 9.52 ± 0.34 | 9.46 ± 0.25 | 9.77 ± 0.46 |
234
  | Iris Xe 96 EU | DirectML | ″ | ″ | ✗² | ✗² | ✗² | ✗² |
235
+ | Iris Xe 80 EU iGPU (Raptor Lake-U) | WebGPU | \~3 TFLOPS | \~29 GB/s sh. | 33.4 ± 2.8¹ | ✗¹ | ✗¹ | ✗¹ |
236
  | Iris Xe 80 EU | DirectML | ″ | ″ | ✗³ | ✗³ | ✗³ | ✗³ |
237
 
238
+ ¹ **WebGPU / Iris Xe 80 EU (8 GB RAM):** peaked at **\~3934 MB — essentially the entire \~3940 MB shared budget** (≈50% of 8 GB RAM). Only 8K ran (11 of 15 runs, \~33 s mean) before it OOM'd; every 4K / 1080p / 512 run failed. WebGPU is **unusable at 8 GB RAM**. <br>
239
+ ² **DirectML / Iris Xe 96 EU (16 GB RAM):** out of memory (`8007000E`) **during session creation** (\~12.7 s in) — never ran. DirectML's footprint is **\~16–19 GB** (peak where it ran) vs only **\~8032 MB** shared iGPU budget (≈50% of 16 GB RAM). <br>
240
+ ³ **DirectML / Iris Xe 80 EU (8 GB RAM):** init failed with a GPU **device-hung** error (`887A0007`, "GPU will not respond to more commands") — not strictly OOM, likely a cascade from the preceding WebGPU crash on the same \~3940 MB shared budget.
241
 
242
  **CPU backend** (anchor = cores × clock × AVX width + RAM bandwidth; EP = CPU on every row):
243
 
244
  | CPU (cores/threads, arch) | RAM BW | 8K (s) | 4K (s) | 1080p (s) | 512² (s) |
245
  |---|--:|--:|--:|--:|--:|
246
+ | Core Ultra 7 255H (16C/16T, Arrow Lake-H, AVX2) | \~39 GB/s | 4.57 ± 0.08 | 4.61 ± 0.05 | 4.63 ± 0.12 | 4.65 ± 0.17 |
247
+ | i7-8700K (6C/12T, Coffee Lake, AVX2) | \~25 GB/s | 9.75 ± 0.24 | 9.69 ± 0.10 | 9.67 ± 0.11 | 9.67 ± 0.13 |
248
+ | i7-1165G7 (4C/8T, Tiger Lake, AVX2/512) | \~31 GB/s | 14.6 ± 1.5 | 13.9 ± 0.5 | 13.8 ± 0.3 | 13.9 ± 0.2 |
249
+ | i5-1335U (10C/12T, 15 W, Raptor Lake-U) | \~29 GB/s | 14.3 ± 2.1 | 14.9 ± 1.6 | 15.7 ± 2.0 | 14.3 ± 1.8 |
250
 
251
  (The CPU EP was also run on the RTX 5070 Ti box — that's the i7-8700K row — since the CPU EP ignores the GPU. The 1335U's larger ± is real run-to-run jitter on a 15 W thermally-throttled laptop.)
252
 
253
  **How to read it / rules of thumb:**
254
 
255
+ - **Choose the EP, not just the chip.** Discrete NVIDIA/AMD → **WebGPU** (fastest). Intel **Arc** iGPU → **DirectML** (its WebGPU path is \~5× slower here). Older Intel **Iris Xe** → **WebGPU** if you have ≥16 GB RAM, else **CPU**. No usable GPU → **CPU** (scales with cores × clock × AVX width; the 16-core 255H is \~3× the 4-core 1165G7).
256
+ - **RAM is the gate for iGPUs.** The fp32 model is \~805 MB on disk but a GPU session peaks at **8–20 GB** (VRAM on discrete cards; *shared system RAM* on iGPUs — WebGPU peaked \~8 GB, DirectML \~15–19 GB in our runs). So: iGPU WebGPU needs **≥16 GB system RAM**; iGPU DirectML wants **≥32 GB**; **8 GB machines can only use the CPU**. Discrete GPUs need **≥8 GB VRAM** free.
257
  - **DirectML is hit-or-miss.** It gave the best iGPU time (Arc 140T, 1.6 s) but failed to initialize on three of four machines (driver/VRAM dependent). Always have the CPU EP as a fallback.
258
+ - **Predict GPU time** ≈ scale the RTX 5070 Ti's 0.9 s by its fp16 proxy vs yours, then add \~30–50 % headroom because the model is fp32 and won't fully use tensor cores. (e.g. a \~40 TFLOPS-fp16 card → expect \~2 s; a \~9 TFLOPS iGPU via a *good* runtime → \~1.5–2 s, which matches the Arc 140T on DML.)
259
 
260
  ## Notes
261