Document RealVisXL provenance, conversion, and runtime settings
Browse files
README.md
CHANGED
|
@@ -1,41 +1,62 @@
|
|
| 1 |
---
|
| 2 |
license: openrail++
|
|
|
|
|
|
|
| 3 |
tags:
|
| 4 |
-
- text-to-image
|
| 5 |
-
- stable-diffusion-xl
|
| 6 |
-
- dreamshaper
|
| 7 |
-
- lightning
|
| 8 |
- coreml
|
| 9 |
-
-
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
-
|
| 13 |
-
|
| 14 |
-
library_name: ml-stable-diffusion
|
| 15 |
---
|
| 16 |
|
| 17 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
| 24 |
|
| 25 |
-
-
|
| 26 |
-
-
|
| 27 |
-
-
|
| 28 |
-
-
|
| 29 |
-
-
|
| 30 |
-
-
|
| 31 |
-
- VAE
|
| 32 |
-
-
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
-
-
|
| 39 |
-
- Core ML tooling: [apple/ml-stable-diffusion](https://github.com/apple/ml-stable-diffusion)
|
| 40 |
-
- License: CreativeML Open RAIL++-M; see `LICENSE.md`
|
| 41 |
-
- This mirror imposes no additional restrictions.
|
|
|
|
| 1 |
---
|
| 2 |
license: openrail++
|
| 3 |
+
base_model: SG161222/RealVisXL_V5.0_Lightning
|
| 4 |
+
library_name: coreml
|
| 5 |
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- coreml
|
| 7 |
+
- stable-diffusion-xl
|
| 8 |
+
- text-to-image
|
| 9 |
+
- image-to-image
|
| 10 |
+
- ios
|
| 11 |
+
- localmuse
|
|
|
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# RealVisXL V5.0 Lightning — 6-bit split Core ML
|
| 15 |
+
|
| 16 |
+
Self-contained, compiled SDXL package prepared for on-device use by the LocalMuse iOS app. The repository stays private until the artifact manifest and anonymous downloads are verified.
|
| 17 |
+
|
| 18 |
+
## Provenance
|
| 19 |
+
|
| 20 |
+
- Source UNet: [`SG161222/RealVisXL_V5.0_Lightning`](https://huggingface.co/SG161222/RealVisXL_V5.0_Lightning)
|
| 21 |
+
- Pinned source revision: `f4454158cedaab9f0688c199561d6c92525f3a85`
|
| 22 |
+
- Source fp16 UNet LFS SHA-256: `1143cd2aaf65d24af34b5699d090aed724f6c0978c2ec5a5f56821ccb36260ce`
|
| 23 |
+
- License: CreativeML Open RAIL++-M (`LICENSE.md`), inherited from the source model and SDXL base.
|
| 24 |
+
|
| 25 |
+
The RealVisXL UNet was converted from the pinned source above. The text encoders, VAE encoder/decoder, tokenizer vocabulary, and merges are standard SDXL components reused byte-for-byte from [`LocalMuseAI/coreml-dreamshaper-xl-lightning-6bit`](https://huggingface.co/LocalMuseAI/coreml-dreamshaper-xl-lightning-6bit) revision `736408cecf59b3b81a8b2a900cdd5dbf94c34c2a`. Only the UNet contains RealVisXL-specific weights.
|
| 26 |
+
|
| 27 |
+
## Conversion
|
| 28 |
+
|
| 29 |
+
- Apple converter: [`apple/ml-stable-diffusion`](https://github.com/apple/ml-stable-diffusion) revision `e12202c1f6405b83918b58a5d097cd61e3e1f702`
|
| 30 |
+
- PyTorch `2.4.0`
|
| 31 |
+
- coremltools `8.0`
|
| 32 |
+
- Diffusers `0.30.2`
|
| 33 |
+
- Transformers `4.44.2`
|
| 34 |
+
- huggingface_hub `0.24.6`
|
| 35 |
+
- NumPy `1.23.5`
|
| 36 |
+
- Resolution: fixed `1024 × 1024` (`128 × 128` latent)
|
| 37 |
+
- Classifier-free-guidance batch: `2`
|
| 38 |
+
- Attention implementation: `SPLIT_EINSUM`
|
| 39 |
+
- UNet storage: mixed Float16 and 6-bit palettized weights
|
| 40 |
+
- Minimum Core ML deployment target: iOS 17
|
| 41 |
+
- UNet split into two independently compiled chunks below 1 GiB each
|
| 42 |
|
| 43 |
+
The source model's fp16 UNet was loaded directly without downloading unrelated full-precision weights, converted through Apple's Stable Diffusion implementation, palettized before chunking, bisected with coremltools, and compiled with `xcrun coremlc`.
|
| 44 |
|
| 45 |
+
## Runtime settings
|
| 46 |
|
| 47 |
+
The source model card recommends 5 inference steps, CFG 1–2, and a DPM++ SDE Karras scheduler. LocalMuse uses:
|
| 48 |
|
| 49 |
+
- Steps: `5`
|
| 50 |
+
- CFG: `1.5`
|
| 51 |
+
- Scheduler: second-order DPM-Solver++ multistep
|
| 52 |
+
- Timestep spacing: Karras
|
| 53 |
+
- Batch size: `1` image (the UNet internally uses batch 2 for CFG)
|
| 54 |
+
- UNet: Neural Engine preferred
|
| 55 |
+
- VAE: GPU preferred
|
| 56 |
+
- Minimum device memory exposed by the app: 8 GiB
|
| 57 |
|
| 58 |
+
## Verification
|
| 59 |
|
| 60 |
+
Both chunks compile successfully with Xcode `coremlc`. Their named input/output feature sets and tensor shapes match the SDXL split-UNet contract already used by LocalMuse. The model is distributed as direct authenticated files; the app pins every URL to a repository commit and verifies the exact SHA-256 and byte count before installation.
|
| 61 |
|
| 62 |
+
Real-device image quality and thermal behavior should still be validated before shipping a production App Store build.
|
|
|
|
|
|
|
|
|