1580158
/
automatic
mirror of https://github.com/vladmandic/automatic
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,886 Commits
3 Branches
29 Tags
109 MiB
Commit Graph

127 Commits (4e1f8a2b711784636e27e3db2d9d5eb7ee7170a4)

Author SHA1 Message Date
Vladimir Mandic 1bc7ffed73 lint fixes 2023-10-12 11:40:50 -04:00
Vladimir Mandic 3727bf3d02 new lora handler and remove lyco 2023-10-05 14:30:58 -04:00
Vladimir Mandic 1e70a1ae57 civitai multithreaded downloads 2023-10-03 10:46:00 -04:00
Vladimir Mandic d01f01c981 update upscalers 2023-09-30 09:20:25 -04:00
Vladimir Mandic 7f9013533d better error handling 2023-09-26 12:30:45 -04:00
Vladimir Mandic 38402c8e6e fix new k-diffusers 2023-09-26 09:55:22 -04:00
Vladimir Mandic ff28ac35e0 add actual latent upscalers 2023-09-25 19:46:12 -04:00
Vladimir Mandic 7d5119a5d7 add model metadata 2023-09-24 17:52:21 -04:00
Vladimir Mandic b00c323c31 update changelog 2023-09-21 07:52:09 -04:00
Vladimir Mandic 8ca79ad9b5 refactor upscalers 2023-09-20 13:07:29 -04:00
Vladimir Mandic fc75b5ec41 add wuerstchen model 2023-09-14 11:51:09 -04:00
Vladimir Mandic a1120666d8 fix styles and console logging 2023-09-13 16:50:03 -04:00
Vladimir Mandic 56e041c3b6 improve civitai integration 2023-09-10 18:13:20 -04:00
Vladimir Mandic 5a649f951a skip invalid diffusers model 2023-09-10 15:19:39 -04:00
Vladimir Mandic 250d1bf2fb update hints 2023-09-10 13:05:31 -04:00
Vladimir Mandic 7bda411738 improve styles, better logging 2023-09-09 13:47:46 -04:00
Vladimir Mandic 2187735187 hf quick model scan 2023-09-07 16:54:03 -04:00
Vladimir Mandic 09b4c79f4a add job info to all jobs 2023-09-07 10:40:07 -04:00
Vladimir Mandic 241b47fa72 fix en description style 2023-09-07 09:22:43 -04:00
Vladimir Mandic e9055c7cd8 major refactor 2023-09-04 11:31:29 -04:00
Vladimir Mandic d6f7bb92da add model preview download 2023-08-30 12:51:12 -04:00
Vladimir Mandic 91406bc9a0 update civit downloader 2023-08-26 14:13:56 +00:00
Vladimir Mandic 6a4d4ea5b7 update logging and model hashinh 2023-08-22 18:28:09 +00:00
Vladimir Mandic 53dcf157fb switch abs to relative paths 2023-08-21 08:11:30 +00:00
Vladimir Mandic 5eac99d3f5 optimize diffusers memory handling 2023-08-18 20:41:34 +00:00
Vladimir Mandic 20ba9aa649 update lint 2023-08-15 08:29:04 +02:00
Midcoastal 67f369ed25
A walk() optimization and lint fixes 2023-08-17 16:49:42 -04:00
Midcoastal 89da66207a
I guess I didn't quite understand Lint error C416 
Seems like a rather pedantic check, if you ask me...  Oh well.
 2023-08-16 00:25:27 -04:00
Midcoastal 03363ce866
Resolve Lint Errors: C416 (x2), W291 and F541 2023-08-15 23:20:00 -04:00
Midcoastal 873e864640
Ooof, modelloader changes needed for TI upgrade 2023-08-15 23:09:24 -04:00
Midcoastal 7e4aed7948
Add a FS directory/path cacher 
- Added FS dir/path mechanism to 'modules.modelloader'
- Refactored 'modules.modelloader.load_models' to use the cache
- Refactored 'modules.ui_extra_networks.find_*' methods to use the cache
- Added progress indicator to 'modules.ui_extra_networks.create_html'

The cache, as implimented, will always ensure it is up-to-date (per 'directory_has_changed') and significantly improves loading speed (when used with 'model_loader' and the 'find_*' methods) with large model directories.

Overall load speed tested with ~10k models (mix of checkpoints, loras, lycoris, and embeddings) and ~40k secndary files (images, descriptions, CivitAI Info, etc).  Loading speeds went from ~1 hour and 45 minutes to ~5 minutes.

Confounding variable to loading speeds: this is over a fiber-attached storage device.  Connection is 10gbe full-duplex, remote source has an NVMe raid cache.  Saturation of the network is the norm, but laintency is a factor.  Regardless, small-scale local-storage testing also shows measurable improvements, so this should be a welcome addition.
 2023-08-15 16:07:41 -04:00
Vladimir Mandic 5d5f22e6e1 remove invalid downloaded files 2023-08-12 08:07:13 +00:00
Vladimir Mandic a77fa4c12e add job info 2023-08-09 08:42:26 +00:00
Vladimir Mandic a5f95f4b38 civitai model search and download 2023-08-09 08:01:37 +00:00
Patrick von Platen a9d239ab51 [Diffusers] Make all Kandinsky work 2023-07-25 21:47:49 +00:00
Vladimir Mandic 46709ee391 fix image metadata and diffuser search 2023-07-18 20:41:47 -04:00
Vladimir Mandic debec28be6 rework settings, image-watermark, safe config handling 2023-07-18 14:41:27 -04:00
Vladimir Mandic a7f5e75a35 fix secondary model location 2023-07-15 16:07:10 -04:00
Vladimir Mandic a16eee1504 bugfix release 2023-07-09 10:06:47 -04:00
Vladimir Mandic 89a7ea6a3f overal quality fixes 2023-07-08 09:49:41 -04:00
Vladimir Mandic 8241e33868 major diffusers update 2023-07-03 16:48:03 -04:00
Vladimir Mandic cc685a8729 wip diffusers 2023-07-02 21:07:26 -04:00
Vladimir Mandic a2caafe4df initial diffusers merge into dev 2023-07-02 14:04:54 -04:00
Vladimir Mandic 0ddf613b49 jumbo merge part two 2023-06-14 11:23:02 -04:00
Vladimir Mandic cb307399dd jumbo merge 2023-06-13 11:59:56 -04:00
Patrick von Platen 46d410687e Improve when loading from diffusers 2023-06-05 18:22:17 +00:00
Vladimir Mandic 851d129680 more diffusers work 2023-05-27 15:49:54 -04:00
Vladimir Mandic efd3810860 diffusers merge 2023-05-26 22:42:03 -04:00
Vladimir Mandic 6c66228cde fix models dir 2023-05-18 08:17:49 -04:00
Vladimir Mandic 0ccda9bc8b jumbo patch 2023-05-17 14:15:55 -04:00
Vladimir Mandic 176ac924af make tensorflow optional 2023-04-23 08:11:26 -04:00
Vladimir Mandic 8b1f26324b optional model loader and integrate image info 2023-04-17 15:31:43 -04:00
Vladimir Mandic 58df74cfd5 new middleware handler and ability to restart server on-the-fly 2023-04-16 15:57:00 -04:00
Vladimir Mandic f181885f0c
Merge pull request #57 from AUTOMATIC1111/master 
merge from upstream
 2023-03-25 08:47:00 -04:00
Vladimir Mandic 73135f2c12 optimize model loader 2023-03-24 13:48:26 -04:00
ArrowM 00bd271faf Move `load_file_from_url` 
Why?
one of the internal calls of `load_file_from_url` import cv2, which locks the cv2 site-package, which extensions may (and in our case, is) breaking the installation of some libraries. The base project should be limiting its import of unnecessary libraries when possible during the installation phase.
 2023-03-21 21:13:30 -05:00
Brad Smith d118cb6ea3
use lowercase name for sorting; keep `UpscalerLanczos` and `UpscalerNearest` at the start of the list with `UpscalerNone` 
Co-authored-by: catboxanon <122327233+catboxanon@users.noreply.github.com>
 2023-03-06 13:21:42 -05:00
Brad Smith b15bc73c99
sort upscalers by name 2023-02-24 14:45:08 -05:00
spezialspezial 6524478850
Update modelloader.py 
os.path.getmtime(filename) throws exception later in codepath when meeting broken symlink. For now catch it here early but more checks could be added for robustness.
 2023-02-04 16:52:15 +01:00
AUTOMATIC ce3f639ec8 add more stuff to ignore when creating model from config 
prevent .vae.safetensors files from being listed as stable diffusion models
 2023-01-10 16:51:04 +03:00
AUTOMATIC 2d5a5076bb Make it so that upscalers are not repeated when restarting UI. 2023-01-03 18:38:21 +03:00
AUTOMATIC b6e5edd746 add built-in extension system 
add support for adding upscalers in extensions
move LDSR, ScuNET and SwinIR to built-in extensions
 2022-12-03 18:06:33 +03:00
Tim Patton ac7ecd2d84 Label and load SD .safetensors model files 2022-11-19 14:49:22 -05:00
victorca25 c9bb33dd43 add resrgan 8x, allow use 1x and up to 8x extra models, move BSRGAN model, add nearest 2022-10-30 12:54:06 +01:00
d8ahazard 5d26ba2b4b
Merge branch 'master' into ScuNET 2022-10-02 07:56:22 -05:00
AUTOMATIC 0114057ad6 fix incorrect use of glob in modelloader for #1410 2022-10-02 15:49:42 +03:00
d8ahazard e82ea20299 Optimize model loader 
Child classes only get populated to __subclassess__ when they are imported. We don't actually need to import any of them to webui any more, so clean up webUI imports and make sure loader imports children.

Also, fix command line paths not actually being passed to the scalers.
 2022-09-30 15:26:18 -05:00
d8ahazard 5d52231bcb Logging.... 
Cleanup. Sorry.
 2022-09-30 09:28:48 -05:00
d8ahazard ca87c09c0e Fix recursive model loading 
Ensure we find checkpoints within subdirectories.
 2022-09-30 09:28:28 -05:00
AUTOMATIC d1f098540a remove unwanted formatting/functionality from the PR 2022-09-30 11:42:40 +03:00
AUTOMATIC aec811c5e6 undo upscaler model filename tampering 2022-09-30 10:52:45 +03:00
d8ahazard 435fd2112a Fixes, cleanup. 2022-09-29 19:59:53 -05:00
d8ahazard 0dce0df1ee Holy $hit. 
Yep.

Fix gfpgan_model_arch requirement(s).
Add Upscaler base class, move from images.
Add a lot of methods to Upscaler.
Re-work all the child upscalers to be proper classes.
Add BSRGAN scaler.
Add ldsr_model_arch class, removing the dependency for another repo that just uses regular latent-diffusion stuff.
Add one universal method that will always find and load new upscaler models without having to add new "setup_model" calls. Still need to add command line params, but that could probably be automated.
Add a "self.scale" property to all Upscalers so the scalers themselves can do "things" in response to the requested upscaling size.
Ensure LDSR doesn't get stuck in a longer loop of "upscale/downscale/upscale" as we try to reach the target upscale size.
Add typehints for IDE sanity.
PEP-8 improvements.
Moar.
 2022-09-29 17:46:23 -05:00
d8ahazard 31ad536c33 Remove debugging message 2022-09-27 11:05:25 -05:00
d8ahazard 11875f5863 Use model loader with stable-diffusion too. 
Hook the model loader into the SD_models file.
Add default url/download if checkpoint is not found.
Add matching stablediffusion-models-path argument.
Add message that --ckpt-dir will be removed in the future, but have it pipe to stablediffusion-models-path for now.
Update help strings for models-path args so they're more or less uniform.
Move sd_model "setup" call to webUI with the others.
Ensure "cleanup_models" method moves existing models to the new locations, including SD, and that we aren't deleting folders that still have stuff in them.
 2022-09-27 11:01:13 -05:00
d8ahazard 7d5c29b674 Cleanup existing directories, fixes 2022-09-26 10:27:18 -05:00
d8ahazard 740070ea9c Re-implement universal model loading 2022-09-26 09:29:50 -05:00