Instructions to use xinsir/controlnet-union-sdxl-1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use xinsir/controlnet-union-sdxl-1.0 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("xinsir/controlnet-union-sdxl-1.0", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Inconsistent Results, Do I still have to flip RGB/BGR Channels to use this?
There was a bug before with multiple OpenPose adapters, where in ComfyUI we had to flip RGB -> BGR Channels in order for the pose to work:
I'm currently comparing in ComfyUI, here are tries with Channels flipped R->B B->R
All below is with strength 1.0 and end_percent 0.5
I'm using DW Pose Estimator with this option ON which enables thicker skeleton.
I'm getting very inconsistent results on Pony Models, I set union controlnet type as openpose, but it seems a CN Weight Scaling node with a weight of 0.5 improves results.
After Many Iterations, It seems there is no need to Flip the channels anymore unlike the previous version π
However, Scaling Soft Weights for the CN seems to improve results,