Text-to-Image
MLX
Safetensors
lance
multimodal
apple-silicon
image-generation
video-generation
diffusion
flow-matching
Mixture of Experts
qwen2_5_vl
wan
port
Instructions to use RockTalk/Lance-3B-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use RockTalk/Lance-3B-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Lance-3B-MLX RockTalk/Lance-3B-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
TI2I edit: Lance chat template + 3-component CFG verified. Semantic edits working (bow tie, panther).
Browse files- .gitattributes +1 -0
- README.md +6 -5
- samples/edit_black_panther.png +0 -0
- samples/edit_blue_cat_attempt.png +0 -0
- samples/edit_bowtie.png +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
samples/orange_cat_chair.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
samples/snowy_peak.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
samples/sunset_mountains.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
samples/orange_cat_chair.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
samples/snowy_peak.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
samples/sunset_mountains.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
samples/edit_bowtie.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -39,7 +39,7 @@ The architecture is **Qwen2.5-VL-3B + parallel MoE-gen experts + Wan 2.2 VAE**.
|
|
| 39 |
| MoE-gen routing (per-token attention + MLP + layernorm) | β
Working |
|
| 40 |
| Text-to-video (T2V) | β
Working on [Lance-3B-Video-MLX](https://huggingface.co/RockTalk/Lance-3B-Video-MLX) (verified at T_lat=3, 9 frames @ 256Γ256) |
|
| 41 |
| XβT (image understanding) | β
Working β accurate captioning at ~29 tok/s with KV cache |
|
| 42 |
-
| Image editing (TI2I) |
|
| 43 |
|
| 44 |
## Sample generations
|
| 45 |
|
|
@@ -53,15 +53,16 @@ Verified on M4 Studio (128 GB). 30 steps, CFG=4, 512Γ512:
|
|
| 53 |
| *"a fluffy orange cat sitting on a wooden chair, photorealistic"* |  |
|
| 54 |
| *"a majestic snowy mountain peak with a dramatic blue sky and clouds"* |  |
|
| 55 |
|
| 56 |
-
### TI2I β image editing
|
| 57 |
|
| 58 |
-
End-to-end
|
| 59 |
|
| 60 |
-
|
| 61 |
|
| 62 |
| Input | Instruction | Output |
|
| 63 |
|---|---|---|
|
| 64 |
-
|  | *"
|
|
|
|
| 65 |
|
| 66 |
### XβT β image understanding
|
| 67 |
|
|
|
|
| 39 |
| MoE-gen routing (per-token attention + MLP + layernorm) | β
Working |
|
| 40 |
| Text-to-video (T2V) | β
Working on [Lance-3B-Video-MLX](https://huggingface.co/RockTalk/Lance-3B-Video-MLX) (verified at T_lat=3, 9 frames @ 256Γ256) |
|
| 41 |
| XβT (image understanding) | β
Working β accurate captioning at ~29 tok/s with KV cache |
|
| 42 |
+
| Image editing (TI2I) | β
Working β ViT + VAE dual conditioning, Lance chat template, three-component CFG (cfg_text + cfg_vit). Semantic edits verified (color change, object addition). |
|
| 43 |
|
| 44 |
## Sample generations
|
| 45 |
|
|
|
|
| 53 |
| *"a fluffy orange cat sitting on a wooden chair, photorealistic"* |  |
|
| 54 |
| *"a majestic snowy mountain peak with a dramatic blue sky and clouds"* |  |
|
| 55 |
|
| 56 |
+
### TI2I β image editing
|
| 57 |
|
| 58 |
+
End-to-end edit pipeline: input image β ViT (UND tokens) + VAE-encode (cond latent) β Lance edit-mode chat template β three-component CFG flow-matching β VAE decode.
|
| 59 |
|
| 60 |
+
Three-component CFG (mirrors PT Lance): `v_final = v_tv_uncond + cfg_text * (v_full - v_t_uncond) + cfg_vit * (v_t_uncond - v_tv_uncond)`. CFG settings: `cfg_text=3.0, cfg_vit=1.0`. ~1.5 s/step at 256Β² (three forward passes per step), 24 steps β 37 s.
|
| 61 |
|
| 62 |
| Input | Instruction | Output |
|
| 63 |
|---|---|---|
|
| 64 |
+
|  | *"Add a small red bow tie to the cat."* |  |
|
| 65 |
+
|  | *"Make the cat completely black, like a panther."* |  |
|
| 66 |
|
| 67 |
### XβT β image understanding
|
| 68 |
|
samples/edit_black_panther.png
ADDED
|
samples/edit_blue_cat_attempt.png
DELETED
|
Binary file (91.6 kB)
|
|
|
samples/edit_bowtie.png
ADDED
|
Git LFS Details
|