d420bc792a
Tweak the memory usage formulas for Flux and SD.
2024-08-01 17:53:45 -04:00
d965474aaa
Make ComfyUI split batches a higher priority than weight offload.
2024-08-01 16:39:59 -04:00
1c61361fd2
Fast preview support for Flux.
2024-08-01 16:28:11 -04:00
a6decf1e62
Fix bfloat16 potentially not being enabled on mps.
2024-08-01 16:18:44 -04:00
48eb1399c0
Try to fix mac issue.
2024-08-01 13:41:27 -04:00
d7430a1651
Add a way to load the diffusion model in fp8 with UNETLoader node.
2024-08-01 13:30:51 -04:00
f2b80f95d2
Better Mac support on flux model.
2024-08-01 13:10:50 -04:00
1aa9cf3292
Make lowvram more aggressive on low memory machines.
2024-08-01 12:11:57 -04:00
eb96c3bd82
Fix .sft file loading (they are safetensors files).
2024-08-01 11:32:58 -04:00
5f98de7697
Load flux t5 in fp8 if weights are in fp8.
2024-08-01 11:05:56 -04:00
8d34211a7a
Fix old python versions no longer working.
2024-08-01 09:57:20 -04:00
1589b58d3e
Basic Flux Schnell and Flux Dev model implementation.
2024-08-01 09:49:29 -04:00
7ad574bffd
Mac supports bf16 just make sure you are using the latest pytorch.
2024-08-01 09:42:17 -04:00
e2382b6adb
Make lowvram less aggressive when there are large amounts of free memory.
2024-08-01 03:58:58 -04:00
c24f897352
Fix to get fp8 working on T5 base.
2024-07-31 02:00:19 -04:00
a5991a7aa6
Fix hunyuan dit text encoder weights always being in fp32.
2024-07-31 01:34:57 -04:00
2c038ccef0
Lower CLIP memory usage by a bit.
2024-07-31 01:32:35 -04:00
b85216a3c0
Lower T5 memory usage by a few hundred MB.
2024-07-31 00:52:34 -04:00
82cae45d44
Fix potential issue with non clip text embeddings.
2024-07-30 14:41:13 -04:00
25853d0be8
Use common function for casting weights to input.
2024-07-30 10:49:14 -04:00
79040635da
Remove unnecessary code.
2024-07-30 05:01:34 -04:00
66d35c07ce
Improve artifacts on hydit, auraflow and SD3 on specific resolutions.
...
This breaks seeds for resolutions that are not a multiple of 16 in pixel
resolution by using circular padding instead of reflection padding but
should lower the amount of artifacts when doing img2img at those
resolutions.
2024-07-29 20:48:50 -04:00
4ba7fa0244
Refactor: Move sd2_clip.py to text_encoders folder.
2024-07-28 01:19:20 -04:00
cf4418b806
Don't treat Bert model like CLIP.
...
Bert can accept up to 512 tokens so any prompt with more than 77 should
just be passed to it as is instead of splitting it up like CLIP.
2024-07-26 13:08:12 -04:00
8328a2d8cd
Let hunyuan dit work with all prompt lengths.
2024-07-26 12:11:32 -04:00
afe732bef9
Hunyuan dit can now accept longer prompts.
2024-07-26 11:52:58 -04:00
a9ac56fc0d
Own BertModel implementation that works with lowvram.
2024-07-26 04:47:17 -04:00
25b51b1a8b
Hunyuan DiT lora support.
2024-07-25 22:42:54 -04:00
a5f4292f9f
Basic hunyuan dit implementation. ( #4102 )
...
* Let tokenizers return weights to be stored in the saved checkpoint.
* Basic hunyuan dit implementation.
* Fix some resolutions not working.
* Support hydit checkpoint save.
* Init with right dtype.
* Switch to optimized attention in pooler.
* Fix black images on hunyuan dit.
2024-07-25 18:21:08 -04:00
f87810cd3e
Let tokenizers return weights to be stored in the saved checkpoint.
2024-07-25 10:52:09 -04:00
10c919f4c7
Make it possible to load tokenizer data from checkpoints.
2024-07-24 16:43:53 -04:00
10b43ceea5
Remove duplicate code.
2024-07-24 01:12:59 -04:00
0a4c49c57c
Support MT5.
2024-07-23 15:35:28 -04:00
88ed893034
Allow SPieceTokenizer to load model from a byte string.
2024-07-23 14:17:42 -04:00
334ba48cea
More generic unet prefix detection code.
2024-07-23 14:13:32 -04:00
14764aa2e2
Rename LLAMATokenizer to SPieceTokenizer.
2024-07-22 12:21:45 -04:00
b2c995f623
"auto" type is only relevant to the SetUnionControlNetType node.
2024-07-22 11:30:38 -04:00
4151fbfa8a
Add error message on union controlnet ( #4081 )
2024-07-22 11:27:32 -04:00
95fa9545f1
Only append zero to noise schedule if last sigma isn't zero.
2024-07-20 12:37:30 -04:00
6ab8cad22e
Implement beta sampling scheduler.
...
It is based on: https://arxiv.org/abs/2407.12173
Add "beta" to the list of schedulers and the BetaSamplingScheduler node.
2024-07-19 18:05:09 -04:00
855789403b
support clip-vit-large-patch14-336 ( #4042 )
...
* support clip-vit-large-patch14-336
* support clip-vit-large-patch14-336
2024-07-17 13:12:50 -04:00
6f7869f365
Get clip vision image size from config.
2024-07-17 13:05:38 -04:00
281ad42df4
Fix lowvram union controlnet bug.
2024-07-17 10:16:31 -04:00
c5a48b15bd
Make default hash lib configurable without code changes via CLI argument ( #3947 )
...
* cli_args: Add --duplicate-check-hash-function.
* server.py: compare_image_hash configurable hash function
Uses an argument added in cli_args to specify the type of hashing to default to for duplicate hash checking. Uses an `eval()` to identify the specific hashlib class to utilize, but ultimately safely operates because we have specific options and only those options/choices in the arg parser. So we don't have any unsafe input there.
* Add hasher() to node_helpers
* hashlib selection moved to node_helpers
* default-hashing-function instead of dupe checking hasher
This makes a default-hashing-function option instead of previous selected option.
* Use args.default_hashing_function
* Use safer handling for node_helpers.hasher()
Uses a safer handling method than `eval` to evaluate default hashing function.
* Stray parentheses are evil.
* Indentation fix.
Somehow when I hit save I didn't notice I missed a space to make indentation work proper. Oops!
2024-07-16 18:27:09 -04:00
8270c62530
Add SetUnionControlNetType to set the type of the union controlnet model.
2024-07-16 17:04:53 -04:00
821f93872e
Allow model sampling to set number of timesteps.
2024-07-16 15:18:40 -04:00
99458e8aca
Add `FrontendManager` to manage non-default front-end impl ( #3897 )
...
* Add frontend manager
* Add tests
* nit
* Add unit test to github CI
* Fix path
* nit
* ignore
* Add logging
* Install test deps
* Remove 'stable' keyword support
* Update test
* Add web-root arg
* Rename web-root to front-end-root
* Add test on non-exist version number
* Use repo owner/name to replace hard coded provider list
* Inline cmd args
* nit
* Fix unit test
2024-07-16 11:26:11 -04:00
1305fb294c
Refactor: Move some code to the comfy/text_encoders folder.
2024-07-15 17:36:24 -04:00
7914c47d5a
Quick fix for the promax controlnet.
2024-07-14 10:07:36 -04:00
a3dffc447a
Support AuraFlow Lora and loading model weights in diffusers format.
...
You can load model weights in diffusers format using the UNETLoader node.
2024-07-13 13:51:40 -04:00
29c2e26724
Better tokenizing code for AuraFlow.
2024-07-12 01:15:25 -04:00
8e012043a9
Add a ModelSamplingAuraFlow node to change the shift value.
...
Set the default AuraFlow shift value to 1.73 (sqrt(3)).
2024-07-11 17:57:36 -04:00
9f291d75b3
AuraFlow model implementation.
2024-07-11 16:52:26 -04:00
f45157e3ac
Fix error message never being shown.
2024-07-11 11:46:51 -04:00
5e1fced639
Cleaner support for loading different diffusion model types.
2024-07-11 11:37:31 -04:00
ffe0bb0a33
Remove useless code.
2024-07-10 20:33:12 -04:00
391c1046cf
More flexibility with text encoder return values.
...
Text encoders can now return other values to the CONDITIONING than the cond
and pooled output.
2024-07-10 20:06:50 -04:00
e44fa5667f
Support returning text encoder attention masks.
2024-07-10 19:31:22 -04:00
f1a01c2c7e
Add sampler_pre_cfg_function ( #3979 )
...
* Update samplers.py
* Update model_patcher.py
2024-07-09 16:20:49 -04:00
ade7aa1b0c
Remove useless import.
2024-07-09 11:05:05 -04:00
faa57430b0
Controlnet union model basic implementation.
...
This is only the model code itself, it currently defaults to an empty
embedding [0] * 6 which seems to work better than treating it like a
regular controlnet.
TODO: Add nodes to select the image type.
2024-07-08 23:49:02 -04:00
bb663bcd6c
Rename clip_t5base to t5base for stable audio text encoder.
2024-07-08 08:53:55 -04:00
2dc84d1444
Add a way to set the timestep multiplier in the flow sampling.
2024-07-06 04:06:03 -04:00
ff63893d10
Support other types of T5 models.
2024-07-06 02:42:53 -04:00
4040491149
Better T5xxl detection.
2024-07-06 00:53:33 -04:00
b8e58a9394
Cleanup T5 code a bit.
2024-07-06 00:36:49 -04:00
80c4590998
Allow specifying the padding token for the tokenizer.
2024-07-06 00:06:49 -04:00
ce649d61c0
Allow zeroing out of embeds with unused attention mask.
2024-07-05 23:48:17 -04:00
739b76630e
Remove useless code.
2024-07-04 15:14:13 -04:00
d7484ef30c
Support loading checkpoints with the UNETLoader node.
2024-07-03 11:34:32 -04:00
537f35c7bc
Don't update dict if contiguous.
2024-07-02 20:21:51 -04:00
3f46362d22
fix non-contiguous tensor saving (from channels-last) ( #3932 )
2024-07-02 20:16:33 -04:00
9dd549e253
Add `--no-custom-node` cmd flag ( #3903 )
...
* Add --no-custom-node cmd flag
* nit
2024-07-01 17:54:03 -04:00
05e831697a
Switch to the real cfg++ method in the samplers.
...
The old _pp ones will be updated automatically to the regular ones with 2x
the cfg.
My fault for not checking what the "_pp" samplers actually did.
2024-06-29 11:59:48 -04:00
264caca20e
ControlNetApplySD3 node can now be used to use SD3 controlnets.
2024-06-27 18:43:11 -04:00
f8f7568d03
Basic SD3 controlnet implementation.
...
Still missing the node to properly use it.
2024-06-27 18:43:11 -04:00
66aaa14001
Controlnet refactor.
2024-06-27 18:43:11 -04:00
8ceb5a02a3
Support saving stable audio checkpoint that can be loaded back.
2024-06-27 11:06:52 -04:00
4f9d2b057c
Remove print.
2024-06-27 02:54:15 -04:00
44947e7ad4
Add DEIS order 3 sampler.
...
Order 4 seems to give bad results.
2024-06-26 22:40:05 -04:00
69d710e40f
Implement my alternative take on CFG++ as the euler_pp sampler.
...
Add euler_ancestral_pp which is the ancestral version of euler with the
same modification.
2024-06-25 07:41:52 -04:00
73ca780019
Add SamplerEulerCFG++ node.
...
This node should match the DDIM implementation of CFG++ when "regular" is
selected.
"alternative" is a slightly different take on CFG++
2024-06-23 13:21:18 -04:00
2f360ae898
Support OneTrainer SD3 lora format.
2024-06-22 13:08:04 -04:00
4ef1479dcd
Multi dimension tiled scale function and tiled VAE audio encoding fallback.
2024-06-22 11:57:49 -04:00
1e2839f4d9
More proper tiled audio decoding.
2024-06-20 16:50:31 -04:00
d5efde89b7
Add ipndm_v sampler, works best with the exponential scheduler.
2024-06-20 08:51:49 -04:00
028a583bef
Fix issue with full diffusers SD3 loras.
2024-06-19 22:32:04 -04:00
0d6a57938e
Support loading diffusers SD3 model format with UNETLoader node.
2024-06-19 22:21:18 -04:00
b08a9dd04b
Remove empty line.
2024-06-19 20:20:35 -04:00
eee815ec99
Update sd1_clip.py ( #3684 )
...
Made token instance check more flexible so it also works with integers from numpy arrays or long tensors
2024-06-19 16:42:41 -04:00
e11052afcf
Add ipndm sampler.
2024-06-19 16:32:30 -04:00
3914d5a2ae
Support full SD3 loras.
2024-06-19 10:13:33 -04:00
a45df69570
Basic tiled decoding for audio VAE.
2024-06-17 22:48:23 -04:00
b7c473d1ab
Fix lora keys for SimpleTuner ( #3759 )
2024-06-17 07:55:06 -04:00
6425252c4f
Use fp16 as the default vae dtype for the audio VAE.
2024-06-16 13:12:54 -04:00
8ddc151a4c
Squash depreciation warning on new pytorch.
2024-06-16 13:06:23 -04:00
ca9d300a80
Better estimation for memory usage during audio VAE encoding/decoding.
2024-06-16 11:47:32 -04:00
746a0410d4
Fix VAEEncode with taesd3.
2024-06-16 03:10:04 -04:00
04e8798c37
Improvements to the TAESD3 implementation.
2024-06-16 02:04:24 -04:00
df7db0e027
support TAESD3 ( #3738 )
2024-06-16 02:03:53 -04:00
bb1969cab7
Initial support for the stable audio open model.
2024-06-15 12:14:56 -04:00
1281f933c1
Small optimization.
2024-06-15 02:44:38 -04:00
f2e844e054
Optimize some unneeded if conditions in the sampling code.
2024-06-15 02:26:19 -04:00
0ec513d877
Add a --force-channels-last to inference models in channel last mode.
2024-06-15 01:08:12 -04:00
0e06b370db
Print key names for easier debugging.
2024-06-14 18:18:53 -04:00
5eb98f0092
Exempt IPEX from non_blocking previews fixing segmentation faults. ( #3708 )
2024-06-13 18:51:14 -04:00
ac151ac169
Support SD3 diffusers lora.
2024-06-13 18:26:10 -04:00
37a08a41b3
Support setting weight offsets in weight patcher.
2024-06-13 17:21:26 -04:00
605e64f6d3
Fix lowvram issue.
2024-06-12 10:39:33 -04:00
1ddf512fdc
Don't auto convert clip and vae weights to fp16 when saving checkpoint.
2024-06-12 01:07:58 -04:00
694e0b48e0
SD3 better memory usage estimation.
2024-06-12 00:49:00 -04:00
69c8d6d8a6
Single and dual clip loader nodes support SD3.
...
You can use the CLIPLoader to use the t5xxl only or the DualCLIPLoader to
use CLIP-L and CLIP-G only for sd3.
2024-06-11 23:27:39 -04:00
0e49211a11
Load the SD3 T5xxl model in the same dtype stored in the checkpoint.
2024-06-11 17:03:26 -04:00
5889b7ca0a
Support multiple text encoder configurations on SD3.
2024-06-11 13:14:43 -04:00
9424522ead
Reuse code.
2024-06-11 07:20:26 -04:00
73ce178021
Remove redundancy in mmdit.py ( #3685 )
2024-06-11 06:30:25 -04:00
a82fae2375
Fix bug with cosxl edit model.
2024-06-10 16:00:03 -04:00
8c4a9befa7
SD3 Support.
2024-06-10 14:06:23 -04:00
a5e6a632f9
Support sampling non 2D latents.
2024-06-10 01:31:09 -04:00
742d5720d1
Support zeroing out text embeddings with the attention mask.
2024-06-09 16:51:58 -04:00
6cd8ffc465
Reshape the empty latent image to the right amount of channels if needed.
2024-06-08 02:35:08 -04:00
56333d4850
Use the end token for the text encoder attention mask.
2024-06-07 03:05:23 -04:00
104fcea0c8
Add function to get the list of currently loaded models.
2024-06-05 23:25:16 -04:00
b1fd26fe9e
pytorch xpu should be flash or mem efficient attention?
2024-06-04 17:44:14 -04:00
809cc85a8e
Remove useless code.
2024-06-02 19:23:37 -04:00
b249862080
Add an annoying print to a function I want to remove.
2024-06-01 12:47:31 -04:00
bf3e334d46
Disable non_blocking when --deterministic or directml.
2024-05-30 11:07:38 -04:00
b26da2245f
Fix UnetParams annotation typo ( #3589 )
2024-05-27 19:30:35 -04:00
0920e0e5fe
Remove some unused imports.
2024-05-27 19:08:27 -04:00
ffc4b7c30e
Fix DORA strength.
...
This is a different version of #3298 with more correct behavior.
2024-05-25 02:50:11 -04:00
efa5a711b2
Reduce memory usage when applying DORA: #3557
2024-05-24 23:36:48 -04:00
6c23854f54
Fix OSX latent2rgb previews.
2024-05-22 13:56:28 -04:00
7718ada4ed
Add type annotation UnetWrapperFunction ( #3531 )
...
* Add type annotation UnetWrapperFunction
* nit
* Add types.py
2024-05-22 02:07:27 -04:00
8508df2569
Work around black image bug on Mac 14.5 by forcing attention upcasting.
2024-05-21 16:56:33 -04:00
83d969e397
Disable xformers when tracing model.
2024-05-21 13:55:49 -04:00
1900e5119f
Fix potential issue.
2024-05-20 08:19:54 -04:00
09e069ae6c
Log the pytorch version.
2024-05-20 06:22:29 -04:00
11a2ad5110
Fix controlnet not upcasting on models that have it enabled.
2024-05-19 17:58:03 -04:00
0bdc2b15c7
Cleanup.
2024-05-18 10:11:44 -04:00
98f828fad9
Remove unnecessary code.
2024-05-18 09:36:44 -04:00
19300655dd
Don't automatically switch to lowvram mode on GPUs with low memory.
2024-05-17 00:31:32 -04:00
46daf0a9a7
Add debug options to force on and off attention upcasting.
2024-05-16 04:09:41 -04:00
2d41642716
Fix lowvram dora issue.
2024-05-15 02:47:40 -04:00
ec6f16adb6
Fix SAG.
2024-05-14 18:02:27 -04:00
bb4940d837
Only enable attention upcasting on models that actually need it.
2024-05-14 17:00:50 -04:00
b0ab31d06c
Refactor attention upcasting code part 1.
2024-05-14 12:47:31 -04:00
f509c6fe21
Fix Intel GPU memory allocation accuracy and documentation update. ( #3459 )
...
* Change calculation of memory total to be more accurate, allocated is actually smaller than reserved.
* Update README.md install documentation for Intel GPUs.
2024-05-12 06:36:30 -04:00
fa6dd7e5bb
Fix lowvram issue with saving checkpoints.
...
The previous fix didn't cover the case where the model was loaded in
lowvram mode right before.
2024-05-12 06:13:45 -04:00
49c20cdc70
No longer necessary.
2024-05-12 05:34:43 -04:00
e1489ad257
Fix issue with lowvram mode breaking model saving.
2024-05-11 21:55:20 -04:00
93e876a3be
Remove warnings that confuse people.
2024-05-09 05:29:42 -04:00
cd07340d96
Typo fix.
2024-05-08 18:36:56 -04:00
c61eadf69a
Make the load checkpoint with config function call the regular one.
...
I was going to completely remove this function because it is unmaintainable
but I think this is the best compromise.
The clip skip and v_prediction parts of the configs should still work but
not the fp16 vs fp32.
2024-05-06 20:04:39 -04:00
a56d02efc7
Change torch.xpu to ipex.optimize, xpu device initialization and remove workaround for text node issue from older IPEX. ( #3388 )
2024-05-02 03:26:50 -04:00
f81a6fade8
Fix some edge cases with samplers and arrays with a single sigma.
2024-05-01 17:05:30 -04:00
2aed53c4ac
Workaround xformers bug.
2024-04-30 21:23:40 -04:00
bacce529fb
Add TLS Support ( #3312 )
...
* Add TLS Support
* Add to readme
* Add guidance for windows users on generating certificates
* Add guidance for windows users on generating certificates
* Fix typo
2024-04-30 20:17:02 -04:00
7990ae18c1
Fix error when more cond masks passed in than batch size ( #3353 )
2024-04-26 12:51:12 -04:00
8dc19e40d1
Don't init a VAE model when there are no VAE weights.
2024-04-24 09:20:31 -04:00
c59fe9f254
Support VAE without quant_conv.
2024-04-18 21:05:33 -04:00
719fb2c81d
Add basic PAG node.
2024-04-14 23:49:50 -04:00
258dbc06c3
Fix some memory related issues.
2024-04-14 12:08:58 -04:00
58812ab8ca
Support SDXS 512 model.
2024-04-12 22:12:35 -04:00
831511a1ee
Fix issue with sampling_settings persisting across models.
2024-04-09 23:20:43 -04:00
30abc324c2
Support properly saving CosXL checkpoints.
2024-04-08 00:36:22 -04:00
0a03009808
Fix issue with controlnet models getting loaded multiple times.
2024-04-06 18:38:39 -04:00
38ed2da2dd
Fix typo in lowvram patcher ( #3209 )
2024-04-05 12:02:13 -04:00
1088d1850f
Support for CosXL models.
2024-04-05 10:53:41 -04:00
41ed7e85ea
Fix object_patches_backup not being the same object across clones.
2024-04-05 00:22:44 -04:00
0f5768e038
Fix missing arguments in cfg_function.
2024-04-04 23:38:57 -04:00
1f4fc9ea0c
Fix issue with get_model_object on patched model.
2024-04-04 23:01:02 -04:00
1a0486bb96
Fix model needing to be loaded on GPU to generate the sigmas.
2024-04-04 22:08:49 -04:00
c6bd456c45
Make zero denoise a NOP.
2024-04-04 11:41:27 -04:00
fcfd2bdf8a
Small cleanup.
2024-04-04 11:16:49 -04:00
0542088ef8
Refactor sampler code for more advanced sampler nodes part 2.
2024-04-04 01:26:41 -04:00
57753c964a
Refactor sampling code for more advanced sampler nodes.
2024-04-03 22:09:51 -04:00
6c6a39251f
Fix saving text encoder in fp8.
2024-04-02 11:46:34 -04:00
e6482fbbfc
Refactor calc_cond_uncond_batch into calc_cond_batch.
...
calc_cond_batch can take an arbitrary amount of cond inputs.
Added a calc_cond_uncond_batch wrapper with a warning so custom nodes
won't break.
2024-04-01 18:07:47 -04:00
575acb69e4
IP2P model loading support.
...
This is the code to load the model and inference it with only a text
prompt. This commit does not contain the nodes to properly use it with an
image input.
This supports both the original SD1 instructpix2pix model and the
diffusers SDXL one.
2024-03-31 03:10:28 -04:00
94a5a67c32
Cleanup to support different types of inpaint models.
2024-03-29 14:44:13 -04:00
5d8898c056
Fix some performance issues with weight loading and unloading.
...
Lower peak memory usage when changing model.
Fix case where model weights would be unloaded and reloaded.
2024-03-28 18:04:42 -04:00
327ca1313d
Support SDXS 0.9
2024-03-27 23:58:58 -04:00
ae77590b4e
dora_scale support for lora file.
2024-03-25 18:09:23 -04:00
c6de09b02e
Optimize memory unload strategy for more optimized performance.
2024-03-24 02:36:30 -04:00
0624838237
Add inverse noise scaling function.
2024-03-21 14:49:11 -04:00
5d875d77fe
Fix regression with lcm not working with batches.
2024-03-20 20:48:54 -04:00
4b9005e949
Fix regression with model merging.
2024-03-20 13:56:12 -04:00
c18a203a8a
Don't unload model weights for non weight patches.
2024-03-20 02:27:58 -04:00
150a3e946f
Make LCM sampler use the model noise scaling function.
2024-03-20 01:35:59 -04:00
40e124c6be
SV3D support.
2024-03-18 16:54:13 -04:00
cacb022c4a
Make saved SD1 checkpoints match more closely the official one.
2024-03-18 00:26:23 -04:00
d7897fff2c
Move cascade scale factor from stage_a to latent_formats.py
2024-03-16 14:49:35 -04:00
f2fe635c9f
SamplerDPMAdaptative node to test the different options.
2024-03-15 22:36:10 -04:00
448d9263a2
Fix control loras breaking.
2024-03-14 09:30:21 -04:00
db8b59ecff
Lower memory usage for loras in lowvram mode at the cost of perf.
2024-03-13 20:07:27 -04:00
2a813c3b09
Switch some more prints to logging.
2024-03-11 16:34:58 -04:00
0ed72befe1
Change log levels.
...
Logging level now defaults to info. --verbose sets it to debug.
2024-03-11 13:54:56 -04:00
65397ce601
Replace prints with logging and add --verbose argument.
2024-03-10 12:14:23 -04:00
5f60ee246e
Support loading the sr cascade controlnet.
2024-03-07 01:22:48 -05:00
03e6e81629
Set upscale algorithm to bilinear for stable cascade controlnet.
2024-03-06 02:59:40 -05:00
03e83bb5d0
Support stable cascade canny controlnet.
2024-03-06 02:25:42 -05:00
10860bcd28
Add compression_ratio to controlnet code.
2024-03-05 15:15:20 -05:00
478f71a249
Remove useless check.
2024-03-04 08:51:25 -05:00
12c1080ebc
Simplify differential diffusion code.
2024-03-03 15:34:42 -05:00
727021bdea
Implement Differential Diffusion ( #2876 )
...
* Implement Differential Diffusion
* Cleanup.
* Fix.
* Masks should be applied at full strength.
* Fix colors.
* Register the node.
* Cleaner code.
* Fix issue with getting unipc sampler.
* Adjust thresholds.
* Switch to linear thresholds.
* Only calculate nearest_idx on valid thresholds.
2024-03-03 15:34:13 -05:00
1abf8374ec
utils.set_attr can now be used to set any attribute.
...
The old set_attr has been renamed to set_attr_param.
2024-03-02 17:27:23 -05:00
dce3555339
Add some tesla pascal GPUs to the fp16 working but slower list.
2024-03-02 17:16:31 -05:00
51df846598
Let conditioning specify custom concat conds.
2024-03-02 11:44:06 -05:00
9f71e4b62d
Let model patches patch sub objects.
2024-03-02 11:43:27 -05:00
00425563c0
Cleanup: Use sampling noise scaling function for inpainting.
2024-03-01 14:24:41 -05:00
c62e836167
Move noise scaling to object with sampling math.
2024-03-01 12:54:38 -05:00
cb7c3a2921
Allow image_only_indicator to be None.
2024-02-29 13:11:30 -05:00
b3e97fc714
Koala 700M and 1B support.
...
Use the UNET Loader node to load the unet file to use them.
2024-02-28 12:10:11 -05:00
37a86e4618
Remove duplicate text_projection key from some saved models.
2024-02-28 03:57:41 -05:00
8daedc5bf2
Auto detect playground v2.5 model.
2024-02-27 18:03:03 -05:00
d46583ecec
Playground V2.5 support with ModelSamplingContinuousEDM node.
...
Use ModelSamplingContinuousEDM with edm_playground_v2.5 selected.
2024-02-27 15:12:33 -05:00
1e0fcc9a65
Make XL checkpoints save in a more standard format.
2024-02-27 02:07:40 -05:00
b416be7d78
Make the text projection saved in the checkpoint the right format.
2024-02-27 01:52:23 -05:00
03c47fc0f2
Add a min_length property to tokenizer class.
2024-02-26 21:36:37 -05:00
8ac69f62e5
Make return_projected_pooled setable from the __init__
2024-02-25 14:49:13 -05:00
ca7c310a0e
Support loading old CLIP models saved with CLIPSave.
2024-02-25 08:29:12 -05:00
c2cb8e889b
Always return unprojected pooled output for gligen.
2024-02-25 07:33:13 -05:00
1cb3f6a83b
Move text projection into the CLIP model code.
...
Fix issue with not loading the SSD1B clip correctly.
2024-02-25 01:41:08 -05:00
6533b172c1
Support text encoder text_projection in lora.
2024-02-24 23:50:46 -05:00
1e5f0f66be
Support lora keys with lora_prior_unet_ and lora_prior_te_
2024-02-23 12:21:20 -05:00
e1cb93c383
Fix model and cond transformer options merge
2024-02-23 01:19:43 -07:00
10847dfafe
Cleanup uni_pc inpainting.
...
This causes some small changes to the uni pc inpainting behavior but it
seems to improve results slightly.
2024-02-23 02:39:35 -05:00
18c151b3e3
Add some latent2rgb matrices for previews.
2024-02-20 10:57:24 -05:00
0d0fbabd1d
Pass pooled CLIP to stage b.
2024-02-20 04:24:45 -05:00
c6b7a157ed
Align simple scheduling closer to official stable cascade scheduler.
2024-02-20 04:24:39 -05:00
88f300401c
Enable fp16 by default on mps.
2024-02-19 12:00:48 -05:00
e93cdd0ad0
Remove print.
2024-02-19 11:47:26 -05:00
3711b31dff
Support Stable Cascade in checkpoint format.
2024-02-19 11:20:48 -05:00
d91f45ef28
Some cleanups to how the text encoders are loaded.
2024-02-19 10:46:30 -05:00
a7b5eaa7e3
Forgot to commit this.
2024-02-19 04:25:46 -05:00
3b2e579926
Support loading the Stable Cascade effnet and previewer as a VAE.
...
The effnet can be used to encode images for img2img with Stage C.
2024-02-19 04:10:01 -05:00
dccca1daa5
Fix gligen lowvram mode.
2024-02-18 02:20:23 -05:00
8b60d33bb7
Add ModelSamplingStableCascade to control the shift sampling parameter.
...
shift is 2.0 by default on Stage C and 1.0 by default on Stage B.
2024-02-18 00:55:23 -05:00
6bcf57ff10
Fix attention masks properly for multiple batches.
2024-02-17 16:15:18 -05:00
11e3221f1f
fp8 weight support for Stable Cascade.
2024-02-17 15:27:31 -05:00
f8706546f3
Fix attention mask batch size in some attention functions.
2024-02-17 15:22:21 -05:00
3b9969c1c5
Properly fix attention masks in CLIP with batches.
2024-02-17 12:13:13 -05:00
5b40e7a5ed
Implement shift schedule for cascade stage C.
2024-02-17 11:38:47 -05:00
929e266f3e
Manual cast for bf16 on older GPUs.
2024-02-17 09:01:17 -05:00
6c875d846b
Fix clip attention mask issues on some hardware.
2024-02-17 07:53:52 -05:00
805c36ac9c
Make Stable Cascade work on old pytorch 2.0
2024-02-17 00:42:30 -05:00
f2d1d16f4f
Support Stable Cascade Stage B lite.
2024-02-16 23:41:23 -05:00
0b3c50480c
Make --force-fp32 disable loading models in bf16.
2024-02-16 23:01:54 -05:00
97d03ae04a
StableCascade CLIP model support.
2024-02-16 13:29:04 -05:00
667c92814e
Stable Cascade Stage B.
2024-02-16 13:02:03 -05:00
f83109f09b
Stable Cascade Stage C.
2024-02-16 10:55:08 -05:00
5e06baf112
Stable Cascade Stage A.
2024-02-16 06:30:39 -05:00
aeaeca10bd
Small refactor of is_device_* functions.
2024-02-15 21:10:10 -05:00
38b7ac6e26
Don't init the CLIP model when the checkpoint has no CLIP weights.
2024-02-13 00:01:08 -05:00
7dd352cbd7
Merge branch 'feature_expose_discard_penultimate_sigma' of https://github.com/blepping/ComfyUI
2024-02-11 12:23:30 -05:00
f44225fd5f
Fix infinite while loop being possible in ddim_scheduler
2024-02-09 17:11:34 -06:00
25a4805e51
Add a way to set different conditioning for the controlnet.
2024-02-09 14:13:31 -05:00
a352c021ec
Allow custom samplers to request discard penultimate sigma
2024-02-08 02:24:23 -07:00
c661a8b118
Don't use numpy for calculating sigmas.
2024-02-07 18:52:51 -05:00
236bda2683
Make minimum tile size the size of the overlap.
2024-02-05 01:29:26 -05:00
66e28ef45c
Don't use is_bf16_supported to check for fp16 support.
2024-02-04 20:53:35 -05:00
24129d78e6
Speed up SDXL on 16xx series with fp16 weights and manual cast.
2024-02-04 13:23:43 -05:00
4b0239066d
Always use fp16 for the text encoders.
2024-02-02 10:02:49 -05:00
da7a8df0d2
Put VAE key name in model config.
2024-01-30 02:24:38 -05:00
89507f8adf
Remove some unused imports.
2024-01-25 23:42:37 -05:00
05cd00695a
typo fix - calculate_sigmas_scheduler ( #2619 )
...
self.scheduler -> scheduler_name
Co-authored-by: Lt.Dr.Data <lt.dr.data@gmail.com>
2024-01-23 03:47:01 -05:00
4871a36458
Cleanup some unused imports.
2024-01-21 21:51:22 -05:00
78a70fda87
Remove useless import.
2024-01-19 15:38:05 -05:00
d76a04b6ea
Add unfinished ImageOnlyCheckpointSave node to save a SVD checkpoint.
...
This node is unfinished, SVD checkpoints saved with this node will
work with ComfyUI but not with anything else.
2024-01-17 19:46:21 -05:00
f9e55d8463
Only auto enable bf16 VAE on nvidia GPUs that actually support it.
2024-01-15 03:10:22 -05:00
2395ae740a
Make unclip more deterministic.
...
Pass a seed argument note that this might make old unclip images different.
2024-01-14 17:28:31 -05:00
53c8a99e6c
Make server storage the default.
...
Remove --server-storage argument.
2024-01-11 17:21:40 -05:00
977eda19a6
Don't round noise mask.
2024-01-11 03:29:58 -05:00
10f2609fdd
Add InpaintModelConditioning node.
...
This is an alternative to VAE Encode for inpaint that should work with
lower denoise.
This is a different take on #2501
2024-01-11 03:15:27 -05:00
1a57423d30
Fix issue when using multiple t2i adapters with batched images.
2024-01-10 04:00:49 -05:00
6a7bc35db8
Use basic attention implementation for small inputs on old pytorch.
2024-01-09 13:46:52 -05:00
235727fed7
Store user settings/data on the server and multi user support ( #2160 )
...
* wip per user data
* Rename, hide menu
* better error
rework default user
* store pretty
* Add userdata endpoints
Change nodetemplates to userdata
* add multi user message
* make normal arg
* Fix tests
* Ignore user dir
* user tests
* Changed to default to browser storage and add server-storage arg
* fix crash on empty templates
* fix settings added before load
* ignore parse errors
2024-01-08 17:06:44 -05:00
c6951548cf
Update optimized_attention_for_device function for new functions that
...
support masked attention.
2024-01-07 13:52:08 -05:00
aaa9017302
Add attention mask support to sub quad attention.
2024-01-07 04:13:58 -05:00
0c2c9fbdfa
Support attention mask in split attention.
2024-01-06 13:16:48 -05:00
3ad0191bfb
Implement attention mask on xformers.
2024-01-06 04:33:03 -05:00
8c6493578b
Implement noise augmentation for SD 4X upscale model.
2024-01-03 14:27:11 -05:00
ef4f6037cb
Fix model patches not working in custom sampling scheduler nodes.
2024-01-03 12:16:30 -05:00
a7874d1a8b
Add support for the stable diffusion x4 upscaling model.
...
This is an old model.
Load the checkpoint like a regular one and use the new
SD_4XUpscale_Conditioning node.
2024-01-03 03:37:56 -05:00
2c4e92a98b
Fix regression.
2024-01-02 14:41:33 -05:00
5eddfdd80c
Refactor VAE code.
...
Replace constants with downscale_ratio and latent_channels.
2024-01-02 13:24:34 -05:00
a47f609f90
Auto detect out_channels from model.
2024-01-02 01:50:57 -05:00
79f73a4b33
Remove useless code.
2024-01-02 01:50:29 -05:00
1b103e0cb2
Add argument to run the VAE on the CPU.
2023-12-30 05:49:07 -05:00
12e822c6c8
Use function to calculate model size in model patcher.
2023-12-28 21:46:20 -05:00
e1e322cf69
Load weights that can't be lowvramed to target device.
2023-12-28 21:41:10 -05:00
c782144433
Fix clip vision lowvram mode not working.
2023-12-27 13:50:57 -05:00
f21bb41787
Fix taesd VAE in lowvram mode.
2023-12-26 12:52:21 -05:00
61b3f15f8f
Fix lowvram mode not working with unCLIP and Revision code.
2023-12-26 05:02:02 -05:00
d0165d819a
Fix SVD lowvram mode.
2023-12-24 07:13:18 -05:00
a252963f95
--disable-smart-memory now unloads everything like it did originally.
2023-12-23 04:25:06 -05:00
36a7953142
Greatly improve lowvram sampling speed by getting rid of accelerate.
...
Let me know if this breaks anything.
2023-12-22 14:38:45 -05:00
261bcbb0d9
A few missing comfy ops in the VAE.
2023-12-22 04:05:42 -05:00
9a7619b72d
Fix regression with inpaint model.
2023-12-19 02:32:59 -05:00
571ea8cdcc
Fix SAG not working with cfg 1.0
2023-12-18 17:03:32 -05:00
8cf1daa108
Fix SDXL area composition sometimes not using the right pooled output.
2023-12-18 12:54:23 -05:00
2258f85159
Support stable zero 123 model.
...
To use it use the ImageOnlyCheckpointLoader to load the checkpoint and
the new Stable_Zero123 node.
2023-12-18 03:48:04 -05:00
2f9d6a97ec
Add --deterministic option to make pytorch use deterministic algorithms.
2023-12-17 16:59:21 -05:00
e45d920ae3
Don't resize clip vision image when the size is already good.
2023-12-16 03:06:10 -05:00
13e6d5366e
Switch clip vision to manual cast.
...
Make it use the same dtype as the text encoder.
2023-12-16 02:47:26 -05:00
719fa0866f
Set clip vision model in eval mode so it works without inference mode.
2023-12-15 18:53:08 -05:00
574363a8a6
Implement Perp-Neg
2023-12-16 00:28:16 +05:30
a5056cfb1f
Remove useless code.
2023-12-15 01:28:16 -05:00
329c571993
Improve code legibility.
2023-12-14 11:41:49 -05:00
6c5990f7db
Fix cfg being calculated more than once if sampler_cfg_function.
2023-12-13 20:28:04 -05:00
ba04a87d10
Refactor and improve the sag node.
...
Moved all the sag related code to comfy_extras/nodes_sag.py
2023-12-13 16:11:26 -05:00
6761233e9d
Implement Self-Attention Guidance ( #2201 )
...
* First SAG test
* need to put extra options on the model instead of patcher
* no errors and results seem not-broken
* Use @ashen-uncensored formula, which works better!!!
* Fix a crash when using weird resolutions. Remove an unnecessary UNet call
* Improve comments, optimize memory in blur routine
* SAG works with sampler_cfg_function
2023-12-13 15:52:11 -05:00
b454a67bb9
Support segmind vega model.
2023-12-12 19:09:53 -05:00
824e4935f5
Add dtype parameter to VAE object.
2023-12-12 12:03:29 -05:00
32b7e7e769
Add manual cast to controlnet.
2023-12-12 11:32:42 -05:00
3152023fbc
Use inference dtype for unet memory usage estimation.
2023-12-11 23:50:38 -05:00
77755ab8db
Refactor comfy.ops
...
comfy.ops -> comfy.ops.disable_weight_init
This should make it more clear what they actually do.
Some unused code has also been removed.
2023-12-11 23:27:13 -05:00
b0aab1e4ea
Add an option --fp16-unet to force using fp16 for the unet.
2023-12-11 18:36:29 -05:00
ba07cb748e
Use faster manual cast for fp8 in unet.
2023-12-11 18:24:44 -05:00
57926635e8
Switch text encoder to manual cast.
...
Use fp16 text encoder weights for CPU inference to lower memory usage.
2023-12-10 23:00:54 -05:00
340177e6e8
Disable non blocking on mps.
2023-12-10 01:30:35 -05:00
614b7e731f
Implement GLora.
2023-12-09 18:15:26 -05:00
cb63e230b4
Make lora code a bit cleaner.
2023-12-09 14:15:09 -05:00
174eba8e95
Use own clip vision model implementation.
2023-12-09 11:56:31 -05:00
97015b6b38
Cleanup.
2023-12-08 16:02:08 -05:00
a4ec54a40d
Add linear_start and linear_end to model_config.sampling_settings
2023-12-08 02:49:30 -05:00
9ac0b487ac
Make --gpu-only put intermediate values in GPU memory instead of cpu.
2023-12-08 02:35:45 -05:00
efb704c758
Support attention masking in CLIP implementation.
2023-12-07 02:51:02 -05:00
fbdb14d4c4
Cleaner CLIP text encoder implementation.
...
Use a simple CLIP model implementation instead of the one from
transformers.
This will allow some interesting things that would too hackish to implement
using the transformers implementation.
2023-12-06 23:50:03 -05:00
2db86b4676
Slightly faster lora applying.
2023-12-06 05:13:14 -05:00
1bbd65ab30
Missed this one.
2023-12-05 12:48:41 -05:00
9b655d4fd7
Fix memory issue with control loras.
2023-12-04 21:55:19 -05:00
26b1c0a771
Fix control lora on fp8.
2023-12-04 13:47:41 -05:00
be3468ddd5
Less useless downcasting.
2023-12-04 12:53:46 -05:00
ca82ade765
Use .itemsize to get dtype size for fp8.
2023-12-04 11:52:06 -05:00
31b0f6f3d8
UNET weights can now be stored in fp8.
...
--fp8_e4m3fn-unet and --fp8_e5m2-unet are the two different formats
supported by pytorch.
2023-12-04 11:10:00 -05:00
af365e4dd1
All the unet ops with weights are now handled by comfy.ops
2023-12-04 03:12:18 -05:00
61a123a1e0
A different way of handling multiple images passed to SVD.
...
Previously when a list of 3 images [0, 1, 2] was used for a 6 frame video
they were concated like this:
[0, 1, 2, 0, 1, 2]
now they are concated like this:
[0, 0, 1, 1, 2, 2]
2023-12-03 03:31:47 -05:00
c97be4db91
Support SD2.1 turbo checkpoint.
2023-11-30 19:27:03 -05:00
983ebc5792
Use smart model management for VAE to decrease latency.
2023-11-28 04:58:51 -05:00
c45d1b9b67
Add a function to load a unet from a state dict.
2023-11-27 17:41:29 -05:00
f30b992b18
.sigma and .timestep now return tensors on the same device as the input.
2023-11-27 16:41:33 -05:00
13fdee6abf
Try to free memory for both cond+uncond before inference.
2023-11-27 14:55:40 -05:00
be71bb5e13
Tweak memory inference calculations a bit.
2023-11-27 14:04:16 -05:00
39e75862b2
Fix regression from last commit.
2023-11-26 03:43:02 -05:00
50dc39d6ec
Clean up the extra_options dict for the transformer patches.
...
Now everything in transformer_options gets put in extra_options.
2023-11-26 03:13:56 -05:00
5d6dfce548
Fix importing diffusers unets.
2023-11-24 20:35:29 -05:00
3e5ea74ad3
Make buggy xformers fall back on pytorch attention.
2023-11-24 03:55:35 -05:00
871cc20e13
Support SVD img2vid model.
2023-11-23 19:41:33 -05:00
410bf07771
Make VAE memory estimation take dtype into account.
2023-11-22 18:17:19 -05:00
32447f0c39
Add sampling_settings so models can specify specific sampling settings.
2023-11-22 17:24:00 -05:00
c3ae99a749
Allow controlling downscale and upscale methods in PatchModelAddDownscale.
2023-11-22 03:23:16 -05:00
72741105a6
Remove useless code.
2023-11-21 17:27:28 -05:00
6a491ebe27
Allow model config to preprocess the vae state dict on load.
2023-11-21 16:29:18 -05:00
cd4fc77d5f
Add taesd and taesdxl to VAELoader node.
...
They will show up if both the taesd_encoder and taesd_decoder or taesdxl
model files are present in the models/vae_approx directory.
2023-11-21 12:54:19 -05:00
ce67dcbcda
Make it easy for models to process the unet state dict on load.
2023-11-20 23:17:53 -05:00
d9d8702d8d
percent_to_sigma now returns a float instead of a tensor.
2023-11-18 23:20:29 -05:00
0cf4e86939
Add some command line arguments to store text encoder weights in fp8.
...
Pytorch supports two variants of fp8:
--fp8_e4m3fn-text-enc (the one that seems to give better results)
--fp8_e5m2-text-enc
2023-11-17 02:56:59 -05:00
107e78b1cb
Add support for loading SSD1B diffusers unet version.
...
Improve diffusers model detection.
2023-11-16 23:12:55 -05:00
7e3fe3ad28
Make deep shrink behave like it should.
2023-11-16 15:26:28 -05:00
9f00a18095
Fix potential issues.
2023-11-16 14:59:54 -05:00
7ea6bb038c
Print warning when controlnet can't be applied instead of crashing.
2023-11-16 12:57:12 -05:00
dcec1047e6
Invert the start and end percentages in the code.
...
This doesn't affect how percentages behave in the frontend but breaks
things if you relied on them in the backend.
percent_to_sigma goes from 0 to 1.0 instead of 1.0 to 0 for less confusion.
Make percent 0 return an extremely large sigma and percent 1.0 return a
zero one to fix imprecision.
2023-11-16 04:23:44 -05:00
57eea0efbb
heunpp2 sampler.
2023-11-14 23:50:55 -05:00
728613bb3e
Fix last pr.
2023-11-14 14:41:31 -05:00
ec3d0ab432
Merge branch 'master' of https://github.com/Jannchie/ComfyUI
2023-11-14 14:38:07 -05:00
c962884a5c
Make bislerp work on GPU.
2023-11-14 11:38:36 -05:00
420beeeb05
Clean up and refactor sampler code.
...
This should make it much easier to write custom nodes with kdiffusion type
samplers.
2023-11-14 00:39:34 -05:00
f2e49b1d57
fix: adaptation to older versions of pytroch
2023-11-14 14:32:05 +09:00
94cc718e9c
Add a way to add patches to the input block.
2023-11-14 00:08:12 -05:00
7339479b10
Disable xformers when it can't load properly.
2023-11-13 12:31:10 -05:00
4781819a85
Make memory estimation aware of model dtype.
2023-11-12 04:28:26 -05:00
dd4ba68b6e
Allow different models to estimate memory usage differently.
2023-11-12 04:03:52 -05:00
2c9dba8dc0
sampling_function now has the model object as the argument.
2023-11-12 03:45:10 -05:00
8d80584f6a
Remove useless argument from uni_pc sampler.
2023-11-12 01:25:33 -05:00
248aa3e563
Fix bug.
2023-11-11 12:20:16 -05:00
4a8a839b40
Add option to use in place weight updating in ModelPatcher.
2023-11-11 01:11:12 -05:00
412d3ff57d
Refactor.
2023-11-11 01:11:06 -05:00
58d5d71a93
Working RescaleCFG node.
...
This was broken because of recent changes so I fixed it and moved it from
the experiments repo.
2023-11-10 20:52:10 -05:00
3e0033ef30
Fix model merge bug.
...
Unload models before getting weights for model patching.
2023-11-10 03:19:05 -05:00
002aefa382
Support lcm models.
...
Use the "lcm" sampler to sample them, you also have to use the
ModelSamplingDiscrete node to set them as lcm models to use them properly.
2023-11-09 18:30:22 -05:00
ec12000136
Add support for full diff lora keys.
2023-11-08 22:05:31 -05:00
064d7583eb
Add a CONDConstant for passing non tensor conds to unet.
2023-11-08 01:59:09 -05:00
794dd2064d
Fix typo.
2023-11-07 23:41:55 -05:00
0a6fd49a3e
Print leftover keys when using the UNETLoader.
2023-11-07 22:15:55 -05:00
fe40109b57
Fix issue with object patches not being copied with patcher.
2023-11-07 22:15:15 -05:00
a527d0c795
Code refactor.
2023-11-07 19:33:40 -05:00
2a23ba0b8c
Fix unet ops not entirely on GPU.
2023-11-07 04:30:37 -05:00
844dbf97a7
Add: advanced->model->ModelSamplingDiscrete node.
...
This allows changing the sampling parameters of the model (eps or vpred)
or set the model to use zsnr.
2023-11-07 03:28:53 -05:00
656c0b5d90
CLIP code refactor and improvements.
...
More generic clip model class that can be used on more types of text
encoders.
Don't apply weighting algorithm when weight is 1.0
Don't compute an empty token output when it's not needed.
2023-11-06 14:17:41 -05:00
b3fcd64c6c
Make SDTokenizer class work with more types of tokenizers.
2023-11-06 01:09:18 -05:00
7e455adc07
fix unet_wrapper_function name in ModelPatcher
2023-11-05 17:11:44 +08:00
1ffa8858e7
Move model sampling code to comfy/model_sampling.py
2023-11-04 01:32:23 -04:00
ae2acfc21b
Don't convert Nan to zero.
...
Converting Nan to zero is a bad idea because it makes it hard to tell when
something went wrong.
2023-11-03 13:13:15 -04:00
d2e27b48f1
sampler_cfg_function now gets the noisy output as argument again.
...
This should make things that use sampler_cfg_function behave like before.
Added an input argument for those that want the denoised output.
This means you can calculate the x0 prediction of the model by doing:
(input - cond) for example.
2023-11-01 21:24:08 -04:00
2455aaed8a
Allow model or clip to be None in load_lora_for_models.
2023-11-01 20:27:20 -04:00
ecb80abb58
Allow ModelSamplingDiscrete to be instantiated without a model config.
2023-11-01 19:13:03 -04:00
e73ec8c4da
Not used anymore.
2023-11-01 00:01:30 -04:00
111f1b5255
Fix some issues with sampling precision.
2023-10-31 23:49:29 -04:00
7c0f255de1
Clean up percent start/end and make controlnets work with sigmas.
2023-10-31 22:14:32 -04:00
a268a574fa
Remove a bunch of useless code.
...
DDIM is the same as euler with a small difference in the inpaint code.
DDIM uses randn_like but I set a fixed seed instead.
I'm keeping it in because I'm sure if I remove it people are going to
complain.
2023-10-31 18:11:29 -04:00
1777b54d02
Sampling code changes.
...
apply_model in model_base now returns the denoised output.
This means that sampling_function now computes things on the denoised
output instead of the model output. This should make things more consistent
across current and future models.
2023-10-31 17:33:43 -04:00
c837a173fa
Fix some memory issues in sub quad attention.
2023-10-30 15:30:49 -04:00
125b03eead
Fix some OOM issues with split attention.
2023-10-30 13:14:11 -04:00
a12cc05323
Add --max-upload-size argument, the default is 100MB.
2023-10-29 03:55:46 -04:00
2a134bfab9
Fix checkpoint loader with config.
2023-10-27 22:13:55 -04:00
e60ca6929a
SD1 and SD2 clip and tokenizer code is now more similar to the SDXL one.
2023-10-27 15:54:04 -04:00
6ec3f12c6e
Support SSD1B model and make it easier to support asymmetric unets.
2023-10-27 14:45:15 -04:00
434ce25ec0
Restrict loading embeddings from embedding folders.
2023-10-27 02:54:13 -04:00
723847f6b3
Faster clip image processing.
2023-10-26 01:53:01 -04:00
a373367b0c
Fix some OOM issues with split and sub quad attention.
2023-10-25 20:17:28 -04:00
7fbb217d3a
Fix uni_pc returning noisy image when steps <= 3
2023-10-25 16:08:30 -04:00
3783cb8bfd
change 'c_adm' to 'y' in ControlNet.get_control
2023-10-25 08:24:32 -05:00
d1d2fea806
Pass extra conds directly to unet.
2023-10-25 00:07:53 -04:00
036f88c621
Refactor to make it easier to add custom conds to models.
2023-10-24 23:31:12 -04:00
3fce8881ca
Sampling code refactor to make it easier to add more conds.
2023-10-24 03:38:41 -04:00
8594c8be4d
Empty the cache when torch cache is more than 25% free mem.
2023-10-22 13:58:12 -04:00
8b65f5de54
attention_basic now works with hypertile.
2023-10-22 03:59:53 -04:00
e6bc42df46
Make sub_quad and split work with hypertile.
2023-10-22 03:51:29 -04:00
a0690f9df9
Fix t2i adapter issue.
2023-10-21 20:31:24 -04:00
9906e3efe3
Make xformers work with hypertile.
2023-10-21 13:23:03 -04:00
4185324a1d
Fix uni_pc sampler math. This changes the images this sampler produces.
2023-10-20 04:16:53 -04:00
e6962120c6
Make sure cond_concat is on the right device.
2023-10-19 01:14:25 -04:00
45c972aba8
Refactor cond_concat into conditioning.
2023-10-18 20:36:58 -04:00
430a8334c5
Fix some potential issues.
2023-10-18 19:48:36 -04:00
782a24fce6
Refactor cond_concat into model object.
2023-10-18 16:48:37 -04:00
0d45a565da
Fix memory issue related to control loras.
...
The cleanup function was not getting called.
2023-10-18 02:43:01 -04:00
d44a2de49f
Make VAE code closer to sgm.
2023-10-17 15:18:51 -04:00
23680a9155
Refactor the attention stuff in the VAE.
2023-10-17 03:19:29 -04:00
c8013f73e5
Add some Quadro cards to the list of cards with broken fp16.
2023-10-16 16:48:46 -04:00
bb064c9796
Add a separate optimized_attention_masked function.
2023-10-16 02:31:24 -04:00
fd4c5f07e7
Add a --bf16-unet to test running the unet in bf16.
2023-10-13 14:51:10 -04:00
9a55dadb4c
Refactor code so model can be a dtype other than fp32 or fp16.
2023-10-13 14:41:17 -04:00
88733c997f
pytorch_attention_enabled can now return True when xformers is enabled.
2023-10-11 21:30:57 -04:00
20d3852aa1
Pull some small changes from the other repo.
2023-10-11 20:38:48 -04:00
ac7d8cfa87
Allow attn_mask in attention_pytorch.
2023-10-11 20:38:48 -04:00
1a4bd9e9a6
Refactor the attention functions.
...
There's no reason for the whole CrossAttention object to be repeated when
only the operation in the middle changes.
2023-10-11 20:38:48 -04:00
8cc75c64ff
Let unet wrapper functions have .to attributes.
2023-10-11 01:34:38 -04:00
5e885bd9c8
Cleanup.
2023-10-10 21:46:53 -04:00
851bb87ca9
Merge branch 'taesd_safetensors' of https://github.com/mochiya98/ComfyUI
2023-10-10 21:42:35 -04:00
9eb621c95a
Supports TAESD models in safetensors format
2023-10-10 13:21:44 +09:00
d1a0abd40b
Merge branch 'input-directory' of https://github.com/jn-jairo/ComfyUI
2023-10-09 01:53:29 -04:00
72188dffc3
load_checkpoint_guess_config can now optionally output the model.
2023-10-06 13:48:18 -04:00
63e5fd1790
Option to input directory
2023-10-04 19:45:15 -03:00
9bfec2bdbf
Fix quality loss due to low precision
2023-10-04 15:40:59 +02:00
0f17993d05
fix: typo in extra sampler
2023-09-29 06:09:59 +03:00
66756de100
Add SamplerDPMPP_2M_SDE node.
2023-09-28 21:56:23 -04:00
71713888c4
Print missing VAE keys.
2023-09-28 00:54:57 -04:00
d234ca558a
Add missing samplers to KSamplerSelect.
2023-09-28 00:17:03 -04:00
1adcc4c3a2
Add a SamplerCustom Node.
...
This node takes a list of sigmas and a sampler object as input.
This lets people easily implement custom schedulers and samplers as nodes.
More nodes will be added to it in the future.
2023-09-27 22:21:18 -04:00
bf3fc2f1b7
Refactor sampling related code.
2023-09-27 16:45:22 -04:00
fff491b032
Model patches can now know which batch is positive and negative.
2023-09-27 12:04:07 -04:00
1d6dd83184
Scheduler code refactor.
2023-09-26 17:07:07 -04:00
446caf711c
Sampling code refactor.
2023-09-26 13:45:15 -04:00
76cdc809bf
Support more controlnet models.
2023-09-23 18:47:46 -04:00
ae87543653
Merge branch 'cast_intel' of https://github.com/simonlui/ComfyUI
2023-09-23 00:57:17 -04:00
eec449ca8e
Allow Intel GPUs to LoRA cast on GPU since it supports BF16 natively.
2023-09-22 21:11:27 -07:00
afa2399f79
Add a way to set output block patches to modify the h and hsp.
2023-09-22 20:26:47 -04:00
492db2de8d
Allow having a different pooled output for each image in a batch.
2023-09-21 01:14:42 -04:00
1cdfb3dba4
Only do the cast on the device if the device supports it.
2023-09-20 17:52:41 -04:00
7c9a92f552
Don't depend on torchvision.
2023-09-19 13:12:47 -04:00
2b6b178173
Added support for lanczos scaling
2023-09-19 10:40:38 +02:00
b92bf8196e
Do lora cast on GPU instead of CPU for higher performance.
2023-09-18 23:04:49 -04:00
321c5fa295
Enable pytorch attention by default on xpu.
2023-09-17 04:09:19 -04:00
61b1f67734
Support models without previews.
2023-09-16 12:59:54 -04:00
43d4935a1d
Add cond_or_uncond array to transformer_options so hooks can check what is
...
cond and what is uncond.
2023-09-15 22:21:14 -04:00
415abb275f
Add DDPM sampler.
2023-09-15 19:22:47 -04:00
94e4fe39d8
This isn't used anywhere.
2023-09-15 12:03:03 -04:00
44361f6344
Support for text encoder models that need attention_mask.
2023-09-15 02:02:05 -04:00
0d8f376446
Set last layer on SD2.x models uses the proper indexes now.
...
Before I had made the last layer the penultimate layer because some
checkpoints don't have them but it's not consistent with the others models.
TLDR: for SD2.x models only: CLIPSetLastLayer -1 is now -2.
2023-09-14 20:28:22 -04:00
0966d3ce82
Don't run text encoders on xpu because there are issues.
2023-09-14 12:16:07 -04:00
3039b08eb1
Only parse command line args when main.py is called.
2023-09-13 11:38:20 -04:00
ed58730658
Don't leave very large hidden states in the clip vision output.
2023-09-12 15:09:10 -04:00
fb3b728203
Fix issue where autocast fp32 CLIP gave different results from regular.
2023-09-11 21:49:56 -04:00
7d401ed1d0
Add ldm format support to UNETLoader.
2023-09-11 16:36:50 -04:00
e85be36bd2
Add a penultimate_hidden_states to the clip vision output.
2023-09-08 14:06:58 -04:00
1e6b67101c
Support diffusers format t2i adapters.
2023-09-08 11:36:51 -04:00
326577d04c
Allow cancelling of everything with a progress bar.
2023-09-07 23:37:03 -04:00
f88f7f413a
Add a ConditioningSetAreaPercentage node.
2023-09-06 03:28:27 -04:00
1938f5c5fe
Add a force argument to soft_empty_cache to force a cache empty.
2023-09-04 00:58:18 -04:00
7746bdf7b0
Merge branch 'generalize_fixes' of https://github.com/simonlui/ComfyUI
2023-09-04 00:43:11 -04:00
2da73b7073
Revert changes in comfy/ldm/modules/diffusionmodules/util.py, which is unused.
2023-09-02 20:07:52 -07:00
a74c5dbf37
Move some functions to utils.py
2023-09-02 22:33:37 -04:00
4a0c4ce4ef
Some fixes to generalize CUDA specific functionality to Intel or other GPUs.
2023-09-02 18:22:10 -07:00
77a176f9e0
Use common function to reshape batch to.
2023-09-02 03:42:49 -04:00
7931ff0fd9
Support SDXL inpaint models.
2023-09-01 15:22:52 -04:00
0e3b641172
Remove xformers related print.
2023-09-01 02:12:03 -04:00
5c363a9d86
Fix controlnet bug.
2023-09-01 02:01:08 -04:00
cfe1c54de8
Fix controlnet issue.
2023-08-31 15:16:58 -04:00
1c012d69af
It doesn't make sense for c_crossattn and c_concat to be lists.
2023-08-31 13:25:00 -04:00
7e941f9f24
Clean up DiffusersLoader node.
2023-08-30 12:57:07 -04:00
18617967e5
Fix error message in model_patcher.py
...
Found while tinkering.
2023-08-30 00:25:04 -07:00
fe4c07400c
Fix "Load Checkpoint with config" node.
2023-08-29 23:58:32 -04:00
f2f5e5dcbb
Support SDXL t2i adapters with 3 channel input.
2023-08-29 16:44:57 -04:00
15adc3699f
Move beta_schedule to model_config and allow disabling unet creation.
2023-08-29 14:22:53 -04:00
bed116a1f9
Remove optimization that caused border.
2023-08-29 11:21:36 -04:00
65cae62c71
No need to check filename extensions to detect shuffle controlnet.
2023-08-28 16:49:06 -04:00
4e89b2c25a
Put clip vision outputs on the CPU.
2023-08-28 16:26:11 -04:00
a094b45c93
Load clipvision model to GPU for faster performance.
2023-08-28 15:29:27 -04:00
1300a1bb4c
Text encoder should initially load on the offload_device not the regular.
2023-08-28 15:08:45 -04:00
f92074b84f
Move ModelPatcher to model_patcher.py
2023-08-28 14:51:31 -04:00
4798cf5a62
Implement loras with norm keys.
2023-08-28 11:20:06 -04:00
b8c7c770d3
Enable bf16-vae by default on ampere and up.
2023-08-27 23:06:19 -04:00
1c794a2161
Fallback to slice attention if xformers doesn't support the operation.
2023-08-27 22:24:42 -04:00
d935ba50c4
Make --bf16-vae work on torch 2.0
2023-08-27 21:33:53 -04:00
a57b0c797b
Fix lowvram model merging.
2023-08-26 11:52:07 -04:00
f72780a7e3
The new smart memory management makes this unnecessary.
2023-08-25 18:02:15 -04:00
c77f02e1c6
Move controlnet code to comfy/controlnet.py
2023-08-25 17:33:04 -04:00
15a7716fa6
Move lora code to comfy/lora.py
2023-08-25 17:11:51 -04:00
ec96f6d03a
Move text_projection to base clip model.
2023-08-24 23:43:48 -04:00
30eb92c3cb
Code cleanups.
2023-08-24 19:39:18 -04:00
51dde87e97
Try to free enough vram for control lora inference.
2023-08-24 17:20:54 -04:00
e3d0a9a490
Fix potential issue with text projection matrix multiplication.
2023-08-24 00:54:16 -04:00
cc44ade79e
Always shift text encoder to GPU when the device supports fp16.
2023-08-23 21:45:00 -04:00
a6ef08a46a
Even with forced fp16 the cpu device should never use it.
2023-08-23 21:38:28 -04:00
00c0b2c507
Initialize text encoder to target dtype.
2023-08-23 21:01:15 -04:00
f081017c1a
Save memory by storing text encoder weights in fp16 in most situations.
...
Do inference in fp32 to make sure quality stays the exact same.
2023-08-23 01:08:51 -04:00
afcb9cb1df
All resolutions now work with t2i adapter for SDXL.
2023-08-22 16:23:54 -04:00
85fde89d7f
T2I adapter SDXL.
2023-08-22 14:40:43 -04:00
cf5ae46928
Controlnet/t2iadapter cleanup.
2023-08-22 01:06:26 -04:00
763b0cf024
Fix control lora not working in fp32.
2023-08-21 20:38:31 -04:00
199d73364a
Fix ControlLora on lowvram.
2023-08-21 00:54:04 -04:00
d08e53de2e
Remove autocast from controlnet code.
2023-08-20 21:47:32 -04:00
0d7b0a4dc7
Small cleanups.
2023-08-20 14:56:47 -04:00
9225465975
Further tuning and fix mem_free_total.
2023-08-20 14:19:53 -04:00
2c096e4260
Add ipex optimize and other enhancements for Intel GPUs based on recent memory changes.
2023-08-20 14:19:51 -04:00
e9469e732d
--disable-smart-memory now disables loading model directly to vram.
2023-08-20 04:00:53 -04:00
c9b562aed1
Free more memory before VAE encode/decode.
2023-08-19 12:13:13 -04:00
b80c3276dc
Fix issue with gligen.
2023-08-18 16:32:23 -04:00
d6e4b342e6
Support for Control Loras.
...
Control loras are controlnets where some of the weights are stored in
"lora" format: an up and a down low rank matrice that when multiplied
together and added to the unet weight give the controlnet weight.
This allows a much smaller memory footprint depending on the rank of the
matrices.
These controlnets are used just like regular ones.
2023-08-18 11:59:51 -04:00
39ac856a33
ReVision support: unclip nodes can now be used with SDXL.
2023-08-18 11:59:36 -04:00
76d53c4622
Add support for clip g vision model to CLIPVisionLoader.
2023-08-18 11:13:29 -04:00
e59fe0537a
Fix referenced before assignment
...
For https://github.com/BlenderNeko/ComfyUI_TiledKSampler/issues/13
2023-08-17 22:30:07 +02:00
be9c5e25bc
Fix issue with not freeing enough memory when sampling.
2023-08-17 15:59:56 -04:00
ac0758a1a4
Fix bug with lowvram and controlnet advanced node.
2023-08-17 13:38:51 -04:00
c28db1f315
Fix potential issues with patching models when saving checkpoints.
2023-08-17 11:07:08 -04:00
3aee33b54e
Add --disable-smart-memory for those that want the old behaviour.
2023-08-17 03:12:37 -04:00
2be2742711
Fix issue with regular torch version.
2023-08-17 01:58:54 -04:00
89a0767abf
Smarter memory management.
...
Try to keep models on the vram when possible.
Better lowvram mode for controlnets.
2023-08-17 01:06:34 -04:00
2c97c30256
Support small diffusers controlnet so both types are now supported.
2023-08-16 12:45:56 -04:00
53f326a3d8
Support diffusers mini controlnets.
2023-08-16 12:28:01 -04:00
58f0c616ed
Fix clip vision issue with old transformers versions.
2023-08-16 11:36:22 -04:00
ae270f79bc
Fix potential issue with batch size and clip vision.
2023-08-16 11:05:11 -04:00
a2ce9655ca
Refactor unclip code.
2023-08-14 23:48:47 -04:00
9cc12c833d
CLIPVisionEncode can now encode multiple images.
2023-08-14 16:54:05 -04:00
0cb6dac943
Remove 3m from PR #1213 because of some small issues.
2023-08-14 00:48:45 -04:00
e244b2df83
Add sgm_uniform scheduler that acts like the default one in sgm.
2023-08-14 00:29:03 -04:00
58c7da3665
Gpu variant of dpmpp_3m_sde. Note: use 3m with exponential or karras.
2023-08-14 00:28:50 -04:00
ba319a34e4
Merge branch 'dpmpp3m' of https://github.com/FizzleDorf/ComfyUI
2023-08-14 00:23:15 -04:00
3cfad03a68
dpmpp 3m + dpmpp 3m sde added
2023-08-13 22:29:04 -04:00
585a062910
Print unet config when model isn't detected.
2023-08-13 01:39:48 -04:00
c8a23ce9e8
Support for yet another lora type based on diffusers.
2023-08-11 13:04:21 -04:00
2bc12d3d22
Add --temp-directory argument to set temp directory.
2023-08-11 05:13:03 -04:00
c20583286f
Support diffuser text encoder loras.
2023-08-10 20:28:28 -04:00
cf10c5592c
Disable calculating uncond when CFG is 1.0
2023-08-09 20:55:03 -04:00
1f0f4cc0bd
Add argument to disable auto launching the browser.
2023-08-07 02:25:12 -04:00
d8e58f0a7e
Detect hint_channels from controlnet.
2023-08-06 14:08:59 -04:00
c5d7593ccf
Support loras in diffusers format.
2023-08-05 01:40:24 -04:00
1ce0d8ad68
Add CMP 30HX card to the nvidia_16_series list.
2023-08-04 12:08:45 -04:00
c99d8002f8
Make sure the pooled output stays at the EOS token with added embeddings.
2023-08-03 20:27:50 -04:00
4a77fcd6ab
Only shift text encoder to vram when CPU cores are under 8.
2023-07-31 00:08:54 -04:00
3cd31d0e24
Lower CPU thread check for running the text encoder on the CPU vs GPU.
2023-07-30 17:18:24 -04:00
2b13939044
Remove some useless code.
2023-07-30 14:13:33 -04:00
95d796fc85
Faster VAE loading.
2023-07-29 16:28:30 -04:00
4b957a0010
Initialize the unet directly on the target device.
2023-07-29 14:51:56 -04:00
c910b4a01c
Remove unused code and torchdiffeq dependency.
2023-07-28 21:32:27 -04:00
1141029a4a
Add --disable-metadata argument to disable saving metadata in files.
2023-07-28 12:31:41 -04:00
fbf5c51c1c
Merge branch 'fix_batch_timesteps' of https://github.com/asagi4/ComfyUI
2023-07-27 16:13:48 -04:00
68be24eead
Remove some prints.
2023-07-27 16:12:43 -04:00
1ea4d84691
Fix timestep ranges when batch_size > 1
2023-07-27 21:14:09 +03:00
5379051d16
Fix diffusers VAE loading.
2023-07-26 18:26:39 -04:00
727588d076
Fix some new loras.
2023-07-25 16:39:15 -04:00
4f9b6f39d1
Fix potential issue with Save Checkpoint.
2023-07-25 00:45:20 -04:00
5f75d784a1
Start is now 0.0 and end is now 1.0 for the timestep ranges.
2023-07-24 18:38:17 -04:00
7ff14b62f8
ControlNetApplyAdvanced can now define when controlnet gets applied.
2023-07-24 17:50:49 -04:00
d191c4f9ed
Add a ControlNetApplyAdvanced node.
...
The controlnet can be applied to the positive or negative prompt only by
connecting it correctly.
2023-07-24 13:35:20 -04:00
0240946ecf
Add a way to set which range of timesteps the cond gets applied to.
2023-07-24 09:25:02 -04:00
22f29d66ca
Try to fix memory issue with lora.
2023-07-22 21:38:56 -04:00
67be7eb81d
Nodes can now patch the unet function.
2023-07-22 17:01:12 -04:00
12a6e93171
Del the right object when applying lora.
2023-07-22 11:25:49 -04:00
78e7958d17
Support controlnet in diffusers format.
2023-07-21 22:58:16 -04:00
09386a3697
Fix issue with lora in some cases when combined with model merging.
2023-07-21 21:27:27 -04:00
58b2364f58
Properly support SDXL diffusers unet with UNETLoader node.
2023-07-21 14:38:56 -04:00
0115018695
Print errors and continue when lora weights are not compatible.
2023-07-20 19:56:22 -04:00
4760c29380
Merge branch 'fix-AttributeError-module-'torch'-has-no-attribute-'mps'' of https://github.com/KarryCharon/ComfyUI
2023-07-20 00:34:54 -04:00
0b284f650b
Fix typo.
2023-07-19 10:20:32 -04:00
e032ca6138
Fix ddim issue with older torch versions.
2023-07-19 10:16:00 -04:00
18885f803a
Add MX450 and MX550 to list of cards with broken fp16.
2023-07-19 03:08:30 -04:00
9ba440995a
It's actually possible to torch.compile the unet now.
2023-07-18 21:36:35 -04:00
51d5477579
Add key to indicate checkpoint is v_prediction when saving.
2023-07-18 00:25:53 -04:00
ff6b047a74
Fix device print on old torch version.
2023-07-17 15:18:58 -04:00
9871a15cf9
Enable --cuda-malloc by default on torch 2.0 and up.
...
Add --disable-cuda-malloc to disable it.
2023-07-17 15:12:10 -04:00
55d0fca9fa
--windows-standalone-build now enables --cuda-malloc
2023-07-17 14:10:36 -04:00
1679abd86d
Add a command line argument to enable backend:cudaMallocAsync
2023-07-17 11:00:14 -04:00
3a150bad15
Only calculate randn in some samplers when it's actually being used.
2023-07-17 10:11:08 -04:00
ee8f8ee07f
Fix regression with ddim and uni_pc when batch size > 1.
2023-07-17 09:35:19 -04:00
3ded1a3a04
Refactor of sampler code to deal more easily with different model types.
2023-07-17 01:22:12 -04:00
5f57362613
Lower lora ram usage when in normal vram mode.
2023-07-16 02:59:04 -04:00
490771b7f4
Speed up lora loading a bit.
2023-07-15 13:25:22 -04:00
50b1180dde
Fix CLIPSetLastLayer not reverting when removed.
2023-07-15 01:41:21 -04:00
comfyanonymous
comfyanonymous
Chenlei Hu
喵哩个咪
Thomas Ward
Extraltodeus
Alex "mcmonkey" Goodwin
Mario Klingemann
Janek Mann
Dr.Lt.Data
Simon Lui
Dango233
JettHu
Garrett Sutula
Jedrzej Kosinski
kk-89
Shiimizu
logtd
blepping
pythongosssss
Hari
Rafie Walker
Jianqi Pan
gameltb
Yukimasa Funaoka
Jairo Correa
City
badayvedat
MoonRide303
Alexopus
FizzleDorf
asagi4