c69af655aa
Uncap cosmos predict2 res and fix mem estimation. ( #8518 )
2025-06-13 07:30:18 -04:00
251f54a2ad
Basic initial support for cosmos predict2 text to image 2B and 14B models. ( #8517 )
2025-06-13 07:05:23 -04:00
1c2d45d2b5
Fix typo in last PR. ( #8144 )
...
More robust model detection for future proofing.
2025-05-15 19:02:19 -04:00
c820ef950d
Add Wan-FUN Camera Control models and Add WanCameraImageToVideo node ( #8013 )
...
* support wan camera models
* fix by ruff check
* change camera_condition type; make camera_condition optional
* support camera trajectory nodes
* fix camera direction
---------
Co-authored-by: Qirui Sun <sunqr0667@126.com>
2025-05-15 19:00:43 -04:00
16417b40d9
Initial ACE-Step model implementation. ( #7972 )
2025-05-07 08:33:34 -04:00
271c9c5b9e
Better mem estimation for the LTXV 13B model. ( #7963 )
2025-05-06 09:52:37 -04:00
08ff5fa08a
Cleanup chroma PR.
2025-04-30 20:57:30 -04:00
4ca3d84277
Support for Chroma - Flux1 Schnell distilled with CFG ( #7355 )
...
* Upload files for Chroma Implementation
* Remove trailing whitespace
* trim more trailing whitespace..oops
* remove unused imports
* Add supported_inference_dtypes
* Set min_length to 0 and remove attention_mask=True
* Set min_length to 1
* get_mdulations added from blepping and minor changes
* Add lora conversion if statement in lora.py
* Update supported_models.py
* update model_base.py
* add uptream commits
* set modelType.FLOW, will cause beta scheduler to work properly
* Adjust memory usage factor and remove unnecessary code
* fix mistake
* reduce code duplication
* remove unused imports
* refactor for upstream sync
* sync chroma-support with upstream via syncbranch patch
* Update sd.py
* Add Chroma as option for the OptimalStepsScheduler node
2025-04-30 20:57:00 -04:00
dbc726f80c
Better vace memory estimation. ( #7875 )
2025-04-29 20:42:00 -04:00
ce22f687cc
Support for WAN VACE preview model. ( #7711 )
...
* Support for WAN VACE preview model.
* Remove print.
2025-04-21 14:40:29 -04:00
9ad792f927
Basic support for hidream i1 model.
2025-04-15 17:35:05 -04:00
3661c833bc
Support the WAN 2.1 fun control models.
...
Use the new WanFunControlToVideo node.
2025-03-26 19:54:54 -04:00
83e839a89b
Native LotusD Implementation ( #7125 )
...
* draft pass at a native comfy implementation of Lotus-D depth and normal est
* fix model_sampling kludges
* fix ruff
---------
Co-authored-by: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
2025-03-21 14:04:15 -04:00
3872b43d4b
A few fixes for the hunyuan3d models.
2025-03-20 04:52:31 -04:00
11f1b41bab
Initial Hunyuan3Dv2 implementation.
...
Supports the multiview, mini, turbo models and VAEs.
2025-03-19 16:52:58 -04:00
11b1f27cb1
Set WAN default compute dtype to fp16.
2025-03-07 04:52:36 -05:00
29a70ca101
Support HunyuanVideo image to video model.
2025-03-06 03:07:15 -05:00
9c9a7f012a
Adjust ltxv memory factor.
2025-03-05 05:16:05 -05:00
7c7c70c400
Refactor skyreels i2v code.
2025-03-04 00:15:45 -05:00
b6fefe686b
Better wan memory estimation.
2025-02-26 07:51:22 -05:00
4ced06b879
WIP support for Wan I2V model.
2025-02-26 01:49:43 -05:00
cb06e9669b
Wan seems to work with fp16.
2025-02-25 21:37:12 -05:00
63023011b9
WIP support for Wan t2v model.
2025-02-25 17:20:35 -05:00
37cd448529
Set the shift for Lumina back to 6.
2025-02-05 14:49:52 -05:00
8ac2dddeed
Lower the default shift of lumina to reduce artifacts.
2025-02-04 06:50:37 -05:00
e5ea112a90
Support Lumina 2 model.
2025-02-04 04:16:30 -05:00
88ceb28e20
Tweak hunyuan memory usage factor.
2025-01-16 06:31:03 -05:00
9d8b6c1f46
More accurate memory estimation for cosmos and hunyuan video.
2025-01-16 03:48:40 -05:00
3aaabb12d4
Implement Cosmos Image/Video to World (Video) diffusion models.
...
Use CosmosImageToVideoLatent to set the input image/video.
2025-01-14 05:14:10 -05:00
2ff3104f70
WIP support for Nvidia Cosmos 7B and 14B text to world (video) models.
2025-01-10 09:14:16 -05:00
4b5bcd8ac4
Closer memory estimation for hunyuan dit model.
2024-12-27 07:37:00 -05:00
ceb50b2cbf
Closer memory estimation for pixart models.
2024-12-27 07:30:09 -05:00
bddb02660c
Add PixArt model support ( #6055 )
...
* PixArt initial version
* PixArt Diffusers convert logic
* pos_emb and interpolation logic
* Reduce duplicate code
* Formatting
* Use optimized attention
* Edit empty token logic
* Basic PixArt LoRA support
* Fix aspect ratio logic
* PixArtAlpha text encode with conds
* Use same detection key logic for PixArt diffusers
2024-12-20 15:25:00 -05:00
d6656b0c0c
Support llama hunyuan video text encoder in scaled fp8 format.
2024-12-17 04:19:22 -05:00
39b1fc4ccc
Adjust used dtypes for hunyuan video VAE and diffusion model.
2024-12-16 23:31:10 -05:00
bda1482a27
Basic Hunyuan Video model support.
2024-12-16 19:35:40 -05:00
d9d7f3c619
Lint all unused variables ( #5989 )
...
* Enable F841
* Autofix
* Remove all unused variable assignment
2024-12-12 17:59:16 -05:00
b7143b74ce
Flux inpaint model does not work in fp16.
2024-11-26 01:33:01 -05:00
5e16f1d24b
Support Lightricks LTX-Video model.
2024-11-22 08:46:39 -05:00
8b275ce5be
Support auto detecting some zsnr anime checkpoints.
2024-11-11 05:34:11 -05:00
669d9e4c67
Set default shift on mochi to 6.0
2024-10-27 22:21:04 -04:00
9ee0a6553a
float16 inference is a bit broken on mochi.
2024-10-27 04:56:40 -04:00
5cbb01bc2f
Basic Genmo Mochi video model support.
...
To use:
"Load CLIP" node with t5xxl + type mochi
"Load Diffusion Model" node with the mochi dit file.
"Load VAE" with the mochi vae file.
EmptyMochiLatentVideo node for the latent.
euler + linear_quadratic in the KSampler node.
2024-10-26 06:54:00 -04:00
83ca891118
Support scaled fp8 t5xxl model.
2024-10-20 22:27:00 -04:00
8f60d093ba
Fix issue.
2024-08-22 10:38:24 -04:00
0f9c2a7822
Try to fix SDXL OOM issue on some configurations.
2024-08-14 23:08:54 -04:00
8115d8cce9
Add Flux fp16 support hack.
2024-08-07 15:08:39 -04:00
2d75df45e6
Flux tweak memory usage.
2024-08-05 21:58:28 -04:00
f123328b82
Load T5 in fp8 if it's in fp8 in the Flux checkpoint.
2024-08-03 12:39:33 -04:00
ea03c9dcd2
Better per model memory usage estimations.
2024-08-02 18:09:24 -04:00
1589b58d3e
Basic Flux Schnell and Flux Dev model implementation.
2024-08-01 09:49:29 -04:00
4ba7fa0244
Refactor: Move sd2_clip.py to text_encoders folder.
2024-07-28 01:19:20 -04:00
a5f4292f9f
Basic hunyuan dit implementation. ( #4102 )
...
* Let tokenizers return weights to be stored in the saved checkpoint.
* Basic hunyuan dit implementation.
* Fix some resolutions not working.
* Support hydit checkpoint save.
* Init with right dtype.
* Switch to optimized attention in pooler.
* Fix black images on hunyuan dit.
2024-07-25 18:21:08 -04:00
1305fb294c
Refactor: Move some code to the comfy/text_encoders folder.
2024-07-15 17:36:24 -04:00
8e012043a9
Add a ModelSamplingAuraFlow node to change the shift value.
...
Set the default AuraFlow shift value to 1.73 (sqrt(3)).
2024-07-11 17:57:36 -04:00
9f291d75b3
AuraFlow model implementation.
2024-07-11 16:52:26 -04:00
8ceb5a02a3
Support saving stable audio checkpoint that can be loaded back.
2024-06-27 11:06:52 -04:00
bb1969cab7
Initial support for the stable audio open model.
2024-06-15 12:14:56 -04:00
69c8d6d8a6
Single and dual clip loader nodes support SD3.
...
You can use the CLIPLoader to use the t5xxl only or the DualCLIPLoader to
use CLIP-L and CLIP-G only for sd3.
2024-06-11 23:27:39 -04:00
0e49211a11
Load the SD3 T5xxl model in the same dtype stored in the checkpoint.
2024-06-11 17:03:26 -04:00
5889b7ca0a
Support multiple text encoder configurations on SD3.
2024-06-11 13:14:43 -04:00
8c4a9befa7
SD3 Support.
2024-06-10 14:06:23 -04:00
bb4940d837
Only enable attention upcasting on models that actually need it.
2024-05-14 17:00:50 -04:00
1088d1850f
Support for CosXL models.
2024-04-05 10:53:41 -04:00
575acb69e4
IP2P model loading support.
...
This is the code to load the model and inference it with only a text
prompt. This commit does not contain the nodes to properly use it with an
image input.
This supports both the original SD1 instructpix2pix model and the
diffusers SDXL one.
2024-03-31 03:10:28 -04:00
327ca1313d
Support SDXS 0.9
2024-03-27 23:58:58 -04:00
40e124c6be
SV3D support.
2024-03-18 16:54:13 -04:00
cacb022c4a
Make saved SD1 checkpoints match more closely the official one.
2024-03-18 00:26:23 -04:00
b3e97fc714
Koala 700M and 1B support.
...
Use the UNET Loader node to load the unet file to use them.
2024-02-28 12:10:11 -05:00
8daedc5bf2
Auto detect playground v2.5 model.
2024-02-27 18:03:03 -05:00
1e0fcc9a65
Make XL checkpoints save in a more standard format.
2024-02-27 02:07:40 -05:00
1cb3f6a83b
Move text projection into the CLIP model code.
...
Fix issue with not loading the SSD1B clip correctly.
2024-02-25 01:41:08 -05:00
3711b31dff
Support Stable Cascade in checkpoint format.
2024-02-19 11:20:48 -05:00
d91f45ef28
Some cleanups to how the text encoders are loaded.
2024-02-19 10:46:30 -05:00
5b40e7a5ed
Implement shift schedule for cascade stage C.
2024-02-17 11:38:47 -05:00
97d03ae04a
StableCascade CLIP model support.
2024-02-16 13:29:04 -05:00
667c92814e
Stable Cascade Stage B.
2024-02-16 13:02:03 -05:00
f83109f09b
Stable Cascade Stage C.
2024-02-16 10:55:08 -05:00
8c6493578b
Implement noise augmentation for SD 4X upscale model.
2024-01-03 14:27:11 -05:00
a7874d1a8b
Add support for the stable diffusion x4 upscaling model.
...
This is an old model.
Load the checkpoint like a regular one and use the new
SD_4XUpscale_Conditioning node.
2024-01-03 03:37:56 -05:00
2258f85159
Support stable zero 123 model.
...
To use it use the ImageOnlyCheckpointLoader to load the checkpoint and
the new Stable_Zero123 node.
2023-12-18 03:48:04 -05:00
b454a67bb9
Support segmind vega model.
2023-12-12 19:09:53 -05:00
c97be4db91
Support SD2.1 turbo checkpoint.
2023-11-30 19:27:03 -05:00
871cc20e13
Support SVD img2vid model.
2023-11-23 19:41:33 -05:00
e60ca6929a
SD1 and SD2 clip and tokenizer code is now more similar to the SDXL one.
2023-10-27 15:54:04 -04:00
6ec3f12c6e
Support SSD1B model and make it easier to support asymmetric unets.
2023-10-27 14:45:15 -04:00
a74c5dbf37
Move some functions to utils.py
2023-09-02 22:33:37 -04:00
7931ff0fd9
Support SDXL inpaint models.
2023-09-01 15:22:52 -04:00
4b957a0010
Initialize the unet directly on the target device.
2023-07-29 14:51:56 -04:00
4f9b6f39d1
Fix potential issue with Save Checkpoint.
2023-07-25 00:45:20 -04:00
3ded1a3a04
Refactor of sampler code to deal more easily with different model types.
2023-07-17 01:22:12 -04:00
b2f03164c7
Prevent the clip_g position_ids key from being saved in the checkpoint.
...
This is to make it match the official checkpoint.
2023-07-12 20:15:02 -04:00
af7a49916b
Support loading unet files in diffusers format.
2023-07-05 17:38:59 -04:00
f81b192944
Add logit scale parameter so it's present when saving the checkpoint.
2023-07-04 23:01:28 -04:00
9b93b920be
Add CheckpointSave node to save checkpoints.
...
The created checkpoints contain workflow metadata that can be loaded by
dragging them on top of the UI or loading them with the "Load" button.
Checkpoints will be saved in fp16 or fp32 depending on the format ComfyUI
is using for inference on your hardware. To force fp32 use: --force-fp32
Anything that patches the model weights like merging or loras will be
saved.
The output directory is currently set to: output/checkpoints but that might
change in the future.
2023-06-26 12:22:27 -04:00
8607c2d42d
Move latent scale factor from VAE to model.
2023-06-23 02:33:31 -04:00
f87ec10a97
Support base SDXL and SDXL refiner models.
...
Large refactor of the model detection and loading code.
2023-06-22 13:03:50 -04:00
comfyanonymous
George0726
comfyanonymous
Silver
thot experiment
City
Chenlei Hu