* Refactor and Improve Core Components
This commit includes several improvements to the core components of the Kohya_ss GUI:
1. **Refactored `check_torch()` in `setup/setup_common.py`**:
* Broke down the function into smaller, more focused functions for better readability and maintainability.
* Simplified the logic for detecting different hardware backends (CUDA, ROCm, Intel OneAPI).
2. **Improved Error Handling in `setup/setup_common.py`**:
* Made error messages more user-friendly and informative by providing context and suggestions.
* Ensured consistent logging of errors.
* Enhanced error reporting for external processes like Git and pip/uv.
3. **Enhanced Documentation for `config.toml`**:
* Updated the "Custom Path Defaults" section in `README.md` with more details on customizing `config.toml`.
* Provided clearer examples of common configurations.
4. **Reviewed and Updated Dependencies**:
* Removed several unused dependencies from `pyproject.toml` (`altair`, `fairscale`, `imagesize`, `invisible-watermark`, `pytorch-lightning`, `voluptuous`).
* Updated various dependencies to their latest stable and compatible versions.
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Add support for custom learning rate scheduler type to the GUI
* Add .webp image extension support to BLIP2 captioning.
* Check for --debug flag for gui command-line args at startup
* Validate GPU ID accelerate input and return error when needed
* Update to latest sd-scripts dev commit
* Fix issue with pip upgrade
* Remove confusing log after command execution.
* piecewise_constant scheduler
* Update to latest sd-scripts dev commit
* fix: fixed docker-compose for passing models via volumes
* Prevent providing the legacy learning_rate if unet or te learning rate is provided
* Fix toml noise offset parameters based on selected type
* Fix adaptive_noise_scale value not properly loading from json config
* Fix prompt.txt location
* Improve "print command" output format
* Use output model name as wandb run name if not provided
* Update sd-scripts dev release
* Bump crate-ci/typos from 1.21.0 to 1.22.9
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.21.0 to 1.22.9.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.21.0...v1.22.9)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump docker/build-push-action from 5 to 6
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Get latest sd3 code
* Adding SD3 GUI elements
* Fix interactivity
* MVP GUI for SD3
* Fix text encoder issue
* Add fork section to readme
* Update sd3 commit
* Merge security-fix
* Update sc-script to latest code
* Auto-detect model type for safetensors files
Automatically tick the checkboxes for v2 and SDXL on the common training UI
and LoRA extract/merge utilities.
* autodetect-modeltype: remove unused lambda inputs
* rework TE1/TE2 learning rate handling for SDXL dreambooth
SDXL dreambooth apparently trains without the text encoders by default,
requiring the `--train_text_encoder` flag to be passed so that the
learning rates for TE1/TE2 are recognized.
The toml handling now permits 0 to be passed as a learning rate in
order to disable training of one or both text encoders.
This behavior aligns with the description given on the GUI.
TE1/TE2 learning rate parameters can be left blank on the GUI to
not pass a value to the training script.
* dreambooth_gui: fix toml value filtering condition
In python3, `0 == False` will evaluate True.
That can cause arg values of 0 to be wrongly eliminated from the toml output.
The conditional must check the type when comparing for False.
* autodetect-modeltype: also do the v2 checkbox in extract_lora
* Update to latest dev branch code
* bring back SDXLConfig accordion for dreambooth gui (#2694)
b-fission <b-fission@users.noreply.github.com>
* Update to latest sd3 branch commit
* Fix merge issue
* Update gradio version
* Update to latest flux.1 code
* Add Flux.1 Model checkbox and detection
* Adding LoRA type "Flux1" to dropdown
* Added Flux.1 parameters to GUI
* Update sd-scripts and requirements
* Add missing Flux.1 GUI parameters
* Update to latest sd-scripts sd3 code
* Fix issue with cache_text_encoder_outputs
* Update to latest sd-scripts flux1 code
* Adding new flux.1 options to GUI
* Update to latest sd-scripts version of flux.1
* Adding guidance_scale option
* Update to latest sd3 flux.1 sd-scripts
* Add dreambooth and finetuning support for flux.1
* Update README
* Fix t5xxl path issue in DB
* add missing fp8_base parameter
* Fix issue with guidance scale not being passed as float for values like 1
* Temporary fir for blockwise_fused_optimizers
* Update to latest sd-scripts Flux.1 code
* Fix blockwise_fused_optimizers typo
* Add mem_eff_save option to GUI for Flux.1
* Added support for Flux.1 LoRA Merge
* Update to latest sd-scripts sd3 branch code
* Add diffusers option to flux.1 merge LoRA utility
* Fix issue with split_mode and train_blocks
* Updating requirements
* Add flux_fused_backward_pass to dreambooth and finetuning
* Update requirements_linux_docker.txt
update accelerate version for linux_docker
* Update to latest sd3 flux code
* Add extract flux lora GUI
* MErged latest sd3 branch code
* Add support for split_qkv
* Add missing network argument for split_qkv
* Add timestep_sampling shift support
* Update to latest sd-scripts flux.1 code
* Add support for fp8_base_unet
* Update requirements as per sd-scripts suggestion
* Upgrade to cu124
* Update IPEX and ROCm
* Fix issue with balancing when folder with name already exist
* Update sd-scripts
* Removed unsupported parameters from flux lora network
* Bump crate-ci/typos from 1.23.6 to 1.24.3
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.3.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.24.3)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update sd-scripts code
* Adding flux_shift option to timestep_sampling
* Update sd-scripts release
* Add support for Train T5-XXL
* Update sd-scripts submodule
* Add support for cpu_offload_checkpointing to GUI
* Force t5xxl_max_token_length to be served as an integer
* Fix typo for flux_shift
* Update to latest sd-scripts code
* Grouping lora parameters
* Validate if lora type is Flux1 when flux1_checkbox is true
* Improve visual sectioning of parameters for lora
* Add dark mode styles
* Missed one color
* Update sd-scripts and add support for t5xxl LR
* Update transformers and wandb module
* Fix issue with new text_encoder_lr parameter syntax
* Add support for lr_warmup_steps override
* Update lr_warmup_steps code
* Removing stable-diffusion-1.5 default model
* Fix for max_train_steps
* Revert some changes
* Preliminary support for Flux1 OFT
* Fix logic typo
* Update sd-scripts
* Add support for Rank for layers
* Update lora_gui.py
Fixed minor typos of "Regularization"
* Update dreambooth_gui.py
Fixed minor typos of "Regularization"
* Update textual_inversion_gui.py
Fixed minor typos of "Regularization"
* Add support for Blocks to train
* Add missing network parms
* Fix issue with old_lr_warmup_steps
* Update sd-scripts
* Add support for ScheduleFree Optimizer Type
* Update sd-scripts
* Update requirements_pytorch_windows.txt
* Update requirements_pytorch_windows.txt
* Update sd-scripts from origin
* Another sd-script update
* Adding support for blocks_to_swap option to gui
* Fix xformers install issue
* feat(docker): mount models folder as a volume
* feat(docker): add models folder to .dockerignore
* Add support for AdEMAMix8bit optimizer
* Bump crate-ci/typos from 1.23.6 to 1.25.0
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.25.0.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.25.0)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fix typo on README.md
* Add new --noverify option to skip requirements validation on startup
* Update startup GUI code
* Update setup code
* Update sd-scripts
* Update sf-scripts
* Update Lycoris support
* Allow to specify tensorboard host via env var TENSORBOARD_HOST
* Update sd-scripts version
* Update sd-scripts release
* Update sd-scripts
* Add --skip_cache_check option to GUI
* Fix requirements issue
* Add support for LyCORIS LoRA when training Flux.1
* Pin huggingface-hub version for gradio 5
* Update sd-scripts
* Add support for --save_last_n_epochs_state
* Update sd-scripts to version with Differential Output Preservation
* Increase maximum flux-lora merge strength to 2
* Update to latest sd-scripts
* Update requirements syntax (for windows)
* Update requirements for linux
* Update torch version and validation output
* Fix typo
* Update README
* Fix validation issue on linux
* Update sd-scripts, improve requirements outputs
* Update requirements_runpod.txt
* Update requirements for onnxruntime-gpu
Needed for compatibility with CUDA 12.
* Update onnxruntime-gpu==1.19.2
* Update sd-scripts release
* Add support for save_last_n_epochs
* Update sd-scripts
* Bump crate-ci/typos from 1.23.6 to 1.26.8 (#2940)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.26.8.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.26.8)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bmaltais <bernard@ducourier.com>
* fix 'cached_download' from 'huggingface_hub' (#2947)
Describe the bug: cannot import name 'cached_download' from 'huggingface_hub'
It's applyed for all platforms
Co-authored-by: bmaltais <bernard@ducourier.com>
* Add support for quiet output for linux setup
* Fix quiet issue
* Update sd-scripts
* Update sd-scripts with blocks_to_swap support
* Make blocks_to_swap visible in LoRA tab
* Fix blocks_to_swap not properly working
* Update sd-scripts and allow python 3.10 to 3.12
* Fix issue with max_train_steps
* Fix max_train_steps_info error
* Reverting all changes for max_train_steps
* Update sd-scripts
* Update sd-scripts
* Update to latest sd-scripts
* Add support for RAdamScheduleFree
* Add support for huber_scale
* Add support for fused_backward_pass for sd3 finetuning
* Add support for prodigyplus.ProdigyPlusScheduleFree
* SD3 LoRA training MVP
* Make blocks_to_swap common
* Add support for sd3 lora disable_mmap_load_safetensors
* Add a bunch of missing SD3 parameters
* Fix clip_l issue for missing path
* Fix train_t5xxl issue
* Fix network_module issue
* Add uniform to weighting_scheme
* Bump crate-ci/typos from 1.23.6 to 1.28.1 (#2996)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.28.1.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.28.1)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bmaltais <bernard@ducourier.com>
* Update README.md (#3031)
* Bump crate-ci/typos from 1.23.6 to 1.29.0 (#3029)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.29.0.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.29.0)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bmaltais <bernard@ducourier.com>
* Update sd-scripts version
* Update setup.sh (#3054)
Enter the current directory before executing setup.sh, otherwise the installer might failed to find rqeuirements.txt
* Removing wrong folder
* Fix issue with SD3 Lora training blocks_to_swap and fused_backward_pass
* Fix dreambooth issue
* Update to lastest sd-scripts code
* Run on novita (#3119) (#3120)
* add run on novita
* adjust position
Co-authored-by: hugo <liyiligang@users.noreply.github.com>
* Bump crate-ci/typos from 1.23.6 to 1.30.0 (#3101)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.30.0.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.23.6...v1.30.0)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bmaltais <bernard@ducourier.com>
* updated prodigyopt to 1.1.2 and removed duplicated row in requirements.txt (#3065)
* fixed names on LR Schedure dropdown (#3064)
* Update to latest sd-scripts version
* fixed names on LR Schedure dropdown (#3064)
* Cleanup venv3
* Fix issue with gradio on new installations
Add support for latest sd-scripts pytorch-optimizer
* Update README for v25.0.0 release
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: b-fission <b-fission@users.noreply.github.com>
Co-authored-by: DevArqSangoi <lucas.sangoi@gmail.com>
Co-authored-by: Кирилл Москвин <retreat.cost@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: b-fission <131207849+b-fission@users.noreply.github.com>
Co-authored-by: eftSharptooth <76253264+eftSharptooth@users.noreply.github.com>
Co-authored-by: Disty0 <disty@disty.xyz>
Co-authored-by: wcole3 <will.cole3@gmail.com>
Co-authored-by: rohitanshu <85547195+iamrohitanshu@users.noreply.github.com>
Co-authored-by: wzgrx <39661556+wzgrx@users.noreply.github.com>
Co-authored-by: Vladimir Sotnikov <vladimir.s@alphakek.ai>
Co-authored-by: bulieme0 <53142287+bulieme@users.noreply.github.com>
Co-authored-by: Nicolas Pereira <41456803+hqnicolas@users.noreply.github.com>
Co-authored-by: ruucm <ruucm.a@gmail.com>
Co-authored-by: CaledoniaProject <CaledoniaProject@users.noreply.github.com>
Co-authored-by: hugo <liyiligang@users.noreply.github.com>
Co-authored-by: Koro <Koronos@users.noreply.github.com>
* Remove library folder, update setup_windows
* Cleanup sd-scripts files
* set sd-scripts release via file
* More cleanup
* Point to sd-scripts python files
* Fix blip captioning working directory
* Fix typo
* Fix issue with verify lora
* Improve git cloning of sd-scripts repo
* Integrate cloning in linux setup script
* Remove unnecessary folder
* Update linux setup
* Removing unnecessary code
* Fix validation issue under linux
* Update tensorboard version linux
* Updating linux requirements
* Update README
* Added submodule for sd-scripts
* Update for proper submodule handling
* Simplify submodule init
* Update comment
* Improve WSL2 LD_LIBRARY_PATH handling
* Update tensorboard handling of logging_dir
* chore(docker): update TensorFlow and TensorBoard versions
- Upgrade TensorFlow version in Dockerfile and requirements to 2.15.0.post1
- Update TensorBoard version in requirements to 2.15.2
Signed-off-by: 陳鈞 <jim60105@gmail.com>
* chore(docker): Add a mount for `sd-scripts` in the Dockerfile during pip install
- Add a mount step for `sd-scripts` in the Dockerfile during pip install.
Signed-off-by: 陳鈞 <jim60105@gmail.com>
* doc(docker): Update readme about docker
* Fix issue with TE LR
* Fix issue with TE LR
* Update sd-scripts to Feb 27 2024 commit
* Update version of bnb for windows
* Update windows setup
* Update windows setup
* Update bnb linux to match windows
* remove height:auto
* update UI
* unify Quick Pick + custom model selection into gr.Dropdown()
* add list_dirs(), list_files()
* use gr.Accordion()
* always show model + training image folder + output name + minimal default settings
* do not show hidden files
* clean up list_*()
* use PYTHONPATH env variable
* add "tool" classname to gr.Button() for sd-webui compatible
* update *_caption_gui.py using create_refresh_buttons()
* update Utilities->Group Images
* update Utilities->Convert model
* update dataset gui
* update Dreambooth/LoRA folders preparation
* update *lora*_gui UI using list_files(), create_refresh_button()
* more fixes for PYTHONPATH
* Minor edits to basic caption and common gui
* Optimising some code
* Fix path issue
* Fix issues with list_images_dir
* Update finetuning image folder label
* Minor updates to UI
---------
Signed-off-by: 陳鈞 <jim60105@gmail.com>
Co-authored-by: 陳鈞 <jim60105@gmail.com>
Co-authored-by: Won-Kyu Park <wkpark@gmail.com>
* Add get_my_logger()
* Use logger instead of print
* Fix log level
* Removed line-breaks for readability
* Use setup_logging()
* Add rich to requirements.txt
* Make simple
* Use logger instead of print
---------
Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com>
bmaltais
Disty0
Kohya S
Yuta Hayashibe
feffy380
Plat
Raaf
Isotr0py
Sam