Flux is awesome! :)
Added 2024-08-01 20:04:27 +0000 UTC
AuraFlow 0.2, AuraSR v2 and Flux? So many new things!
Flux is most certainly worth a try, but both models will benefit from fairly beefy systems with at least 24GB VRAM and 32GB RAM, so beware to go slow if you don't meet the specs.
4 steps, amazing quality... what will we see in a few months time, I wonder?
Links:
https://huggingface.co/fal/AuraFlow-v0.2https://blog.fal.ai/aurasr-v2/
https://github.com/black-forest-labs/flux
https://huggingface.co/black-forest-labs/FLUX.1-schnell
https://comfyanonymous.github.io/ComfyUI_examples/flux/
https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main
https://blog.fal.ai/flux-the-largest-open-sourced-text2img-model-now-available-on-fal/
Flux
1. Download t5xxl_fp16.safetensors & clip_l.safetensors into your ComfyUI/models/clip/ directory you can find them on: this link.
You can use t5xxl_fp8_e4m3fn.safetensors instead for lower memory usage but the fp16 one is recommended if you have more than 32GB ram.
2. Download the vae from here into your ComfyUI/models/vae/ directory.
3. The Flux Schnell diffusion model weights from here into your ComfyUI/models/unet/ directory.
Update
Also added an inpainting example :)
Update
Added a high-res fix example
Update
Longer prompts can be better. Thing is, I can't be bothered. So I added an LLM to do it for me -
The prompt here was simply "Just make a cool picture with the text "PICTURE" somewhere in it, idk", which resulted in the longer version - "In this visually striking image, the bold letters "PICTURE" sprawl across a worn, velvety red canvas like a vintage cinema sign. Emblazoned across two-thirds of the rectangular canvas, the capitalized letters appear in a distressed cream font, their peeling edges evoking feelings of nostalgia. Against a desolate urban landscape where skyscrapers loom in hues of midnight blue and embers orange behind the letter 'P' and 'C', an isolated, dilapidated silver cinema projector casts long, slivering shadows in monochromatic grayscale across dusty cobblestone streets below."
I mean, who can be bothered to type all that, am i right?
For the LLM / node combo I've started with llama3.1 using these two options for local running:
It's an LLM party! - https://github.com/heshengtao/comfyui_LLM_party
Tested locally with ollama - https://github.com/ollama/ollama - using llama3.1
LLM party has loads of options (it really is an LLM party!), so plenty of choice.
Think I'll try Mistral next...
Comments
That's very low spec hardware for flux, so you'd need to go with one of the smaller GGUF models (anything under 8GB). With both low RAM & VRAM, you may also benefit from a larger swapfile (64+GB). Check your system resources as it's running, as the chances are you're running out of RAM.
Nerdy Rodent
2024-10-02 18:42:44 +0000 UTCComfyUI Stops Running When Using Flux I'm encountering an issue with ComfyUI where it stops running whenever I try to use Flux. Here are the details of my setup and what I've tried so far: System Specs RAM: 16GB VRAM: 8GB When attempting to use Flux in ComfyUI, the application stops running completely. This prevents me from completing my workflow. I've tried using both the Schnell and Dev versions of ComfyUI, but the problem persists in both cases. Has anyone else experienced this issue or know of a potential solution? Any help or guidance would be greatly appreciated. Thank you in advance!
ayoub karimi
2024-10-02 13:50:34 +0000 UTC@Tom (3090 here) try toggling Hardware-accelerated GPU scheduling on/off (windows key, then type 'graphics settings') - i've noticed that turning it off (at least for python) can speed things up. It does take a full restart to take effect (eyeroll) - Windows :( -- Edited and removed a bit, Nerdy's article covers it!! thx NR
NetDude
2024-08-20 05:17:27 +0000 UTCOn MS Windows, make sure this is off - https://nvidia.custhelp.com/app/answers/detail/a_id/5490 - you may also want to update Comfy again anyway, as there were some cuda malloc changes impacting some which only got reverted a few hours ago
Nerdy Rodent
2024-08-14 17:49:14 +0000 UTCThanks, yeah I'm using windows. Maybe time to set up Linux then!
Tom Sveen
2024-08-14 00:46:31 +0000 UTCThat should be fine, hardware wise! It could be that you're using Microsoft Windows, which has an automatic "go slow" mode when you run out of VRAM? If it's a standard Linux install, I'd go for another ComfyUI update as there have been some recent changes.
Nerdy Rodent
2024-08-13 16:48:31 +0000 UTCHi Trying the LLM but for some reason it is extremely slow (5 minutes, 30 seconds). The default workflow takes 15-20 seconds. (The LLM portion only takes like 3-5 seconds). I have a 4090 and 64GB Ram. I always use the largest 24GB model, sampler DEIS, Scheduler Normal, 20 steps, 1280x720. Thanks!
Tom Sveen
2024-08-13 06:50:08 +0000 UTCTo get new nodes for ComfyUI, it's handy to install them via ComfyUI Manager - https://github.com/ltdrdata/ComfyUI-Manager
Nerdy Rodent
2024-08-11 22:09:13 +0000 UTCHi! I am definitely a noob and also have some red rectangles corresponding to: LLM_api_loader, show_text_party, and LLM. Any help is very much appreciated.
Speedy2023
2024-08-11 20:49:50 +0000 UTCThat's via https://github.com/pamparamm/ComfyUI-ppm from the other week :)
Nerdy Rodent
2024-08-07 17:32:28 +0000 UTCCurious how do you get ays+ in the basic Scheduler?
KING LIFER
2024-08-07 14:37:03 +0000 UTCIf you click on "update all" first (as you have a very old install), that will give you CLIPTextEncode. For the others, you'll need something called "ComfyUI manager" from https://github.com/ltdrdata/ComfyUI-Manager which you can then use to install any extra noides automatically :)
Nerdy Rodent
2024-08-04 08:22:32 +0000 UTCmight be a noob question but i am unable to get some of the red rectangles working - these - SUNoiseLatent LLM_api_loader show_text_party CLIPTextEncodeFlux
Avinash Kumar
2024-08-04 04:32:22 +0000 UTCI’m very impressed with Flux, best model so far
Lord Banzai
2024-08-04 03:39:17 +0000 UTCHad a little test with inpainting, and it seems fine :)
Nerdy Rodent
2024-08-02 20:53:38 +0000 UTCStill on my to-do list to test as well. Goes up to 2k nicely though...
Nerdy Rodent
2024-08-02 16:23:21 +0000 UTCMan, it would be awesome if you could inpaint with it!
Joseph Baranowski
2024-08-02 15:11:16 +0000 UTCQuestion on AuraSR, I used your AuraFLOW-HRF-US_V0 workflow with it but it didn't upscale the image. I've tried it with device set to default and CPU but no joy. I have the model downloaded for it and the node is picking that up. Any ideas why I can't get it to Uspcale? It's like the workflow is bypassing the node and moving onto the next step. Edit: I should have checked the terminal window, looks like I'm missing some files.
Runebinder
2024-08-02 11:54:43 +0000 UTCI got Flux working (I had to force fp32) and used the Flux_Basic. It is taking about 20 minutes per image...haha. Oh well, it was worth a try.
Lux Elliott
2024-08-02 02:38:58 +0000 UTCWill we, M1 Ultra folks, have a fighting chance? 64GB RAM/VRAM. I'm happy to give it a go, but thought I'd ask first.
Lux Elliott
2024-08-02 00:52:51 +0000 UTCDownloaded flux-dev earlier and was very impressed with the initial results. Downloading Schnell to see what the quality difference is between the 2. So glad I upgraded and got a 3090 last month lol.
Runebinder
2024-08-02 00:23:38 +0000 UTCI'm testing Flux-dev on my 4090 mobile with 16GB of VRAM and 32GB of system RAM. Using the FP16 T5, a 832x1216 image with 20 steps is taking 2 minutes to generate. Not bad at all.
Richard Gates
2024-08-01 20:08:54 +0000 UTC
