Instructions to use Djrango/Qwen2vl-Flux with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Djrango/Qwen2vl-Flux with Diffusers:
pip install -U diffusers transformers accelerate
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline controlnet = ControlNetModel.from_pretrained("Djrango/Qwen2vl-Flux") pipe = StableDiffusionControlNetPipeline.from_pretrained( "black-forest-labs/FLUX.1-dev,Qwen/Qwen2-VL-7B-Instruct", controlnet=controlnet ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
ComfyUI?
node? workflow? proper safetensors file? anyone?
Also interested here
Bumping. No clue what models to use.
在线等 挺急的Wait online, it's urgent.
Hey guys I’ve been trying to see if I could figure it out here’s an update to what I have done and found
- I put in a GitHub feature request to add support on comfy’s issues page
- I downloaded all the files and tried to run it locally to be met with memory issues leaving me unable to run it locally to then port it to ComfyUi
- I had various chats with ChatGPT to understand the code and the changes I would need to do
- I found an existing node and workflow that lets me load the Qwen2-7b-Instruct model the problem is it just outputs text
- I converted the model to a format will work in ComfyUi under the diffusion_models and checkpoints directory as well as unet in q4 and q8
@Kijai would be our last hope and savior, hopefully he can take a break from doing his amazing 3d sculptures and help make a wrapper or a simple node to bridge the existing model loader with a redux style workflow to make this thing work. I’d be happy if we just have variation mode working.
Here's the links to all the supporting stuff I talked about:
Guide & Help Integrating into ComfyUI: https://chatgpt.com/share/674537c1-b728-800f-9381-ce85f916b3ef
Simplification of model.py for variation mode only: https://chatgpt.com/share/67486e40-0858-800f-97e1-c1779bdf600d
CivitAI mode posted in unet, diffuusion_models, checkpoints format: https://civitai.com/models/982213
ComyUI Feature Request: https://github.com/comfyanonymous/ComfyUI/issues/5777
Existing Qwen2 Model Loader for ComfyUII:
Main Page: https://github.com/ZHO-ZHO-ZHO/ComfyUI-Qwen/tree/main
Workflow: https://github.com/ZHO-ZHO-ZHO/ComfyUI-Qwen/blob/main/QWEN2%20WORKFLOWS/QWen2.5%20%2B%20Flux.1%20Dev%20Story%E3%80%90Zho%E3%80%91.json
是我理解错了吗 这不是一个替代T5的东西 而只是一个llm吗