1580158
/
kohya_ss
mirror of https://github.com/bmaltais/kohya_ss
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,966 Commits
16 Branches
178 Tags
68 MiB
Commit Graph

257 Commits (b9d5db8a423a4e321546eb609bfba0c572306d24)

Author SHA1 Message Date
Kohya S 6f80fe17fc fix crashing in saving lora with clipskip 2023-08-08 21:03:16 +09:00
Kohya S c142dadb46 support sai model spec 2023-08-06 21:50:05 +09:00
Kohya S e5f9772a35 fix training textencoder in sdxl not working 2023-08-05 21:22:50 +09:00
Kohya S 9d855091bf make bitsandbytes optional 2023-08-04 22:29:14 +09:00
Kohya S f3be995c28 remove debug print 2023-08-04 08:44:17 +09:00
Kohya S 9d7619d1eb remove debug print 2023-08-04 08:42:54 +09:00
Kohya S c6d52fdea4 Add workaround for clip's bug for pooled output 2023-08-04 08:38:27 +09:00
Kohya S 0636399c8c add adding v-pred like loss for noise pred 2023-07-31 08:23:28 +09:00
Kohya S f61996b425 remove dependency for albumenations 2023-07-30 16:29:53 +09:00
Kohya S 496c3f2732 arbitrary args for diffusers lr scheduler 2023-07-30 14:36:03 +09:00
Kohya S a296654c1b refactor optimizer selection for bnb 2023-07-30 13:43:29 +09:00
Kohya S e6034b7eb6 move releasing cache outside of the loop 2023-07-30 13:30:42 +09:00
青龍聖者@bdsqlsz 9ec70252d0
Add Paged/ adam8bit/lion8bit for Sdxl bitsandbytes 0.39.1 cuda118 on windows (#623) 
* ADD libbitsandbytes.dll for 0.38.1

* Delete libbitsandbytes_cuda116.dll

* Delete cextension.py

* add main.py

* Update requirements.txt for bitsandbytes 0.38.1

* Update README.md for bitsandbytes-windows

* Update README-ja.md  for bitsandbytes 0.38.1

* Update main.py for return cuda118

* Update train_util.py for lion8bit

* Update train_README-ja.md for lion8bit

* Update train_util.py for add DAdaptAdan and DAdaptSGD

* Update train_util.py for DAdaptadam

* Update train_network.py for dadapt

* Update train_README-ja.md for DAdapt

* Update train_util.py for DAdapt

* Update train_network.py for DAdaptAdaGrad

* Update train_db.py for DAdapt

* Update fine_tune.py for DAdapt

* Update train_textual_inversion.py for DAdapt

* Update train_textual_inversion_XTI.py for DAdapt

* Revert "Merge branch 'qinglong' into main"

This reverts commit b65c023083d6d1e8a30eb42eddd603d1aac97650, reversing
changes made to f6fda20caf5e773d56bcfb5c4575c650bb85362b.

* Revert "Update requirements.txt for bitsandbytes 0.38.1"

This reverts commit 83abc60dfaddb26845f54228425b98dd67997528.

* Revert "Delete cextension.py"

This reverts commit 3ba4dfe046874393f2a022a4cbef3628ada35391.

* Revert "Update README.md for bitsandbytes-windows"

This reverts commit 4642c52086b5e9791233007e2fdfd97f832cd897.

* Revert "Update README-ja.md  for bitsandbytes 0.38.1"

This reverts commit fa6d7485ac067ebc49e6f381afdb8dd2f12caa8f.

* Update train_util.py for DAdaptLion

* Update train_README-zh.md for dadaptlion

* Update train_README-ja.md for DAdaptLion

* add DAdatpt V3

* Alignment

* Update train_util.py for experimental

* Update train_util.py V3

* Update train_util.py

* Update requirements.txt

* Update train_README-zh.md

* Update train_README-ja.md

* Update train_util.py fix

* Update train_util.py

* support Prodigy

* add lower

* Update main.py

* support PagedAdamW8bit/PagedLion8bit

* Update requirements.txt

* update for PageAdamW8bit and PagedLion8bit

* Revert

* revert main

* Update train_util.py

* update for bitsandbytes 0.39.1

* Update requirements.txt

* vram leak fix

---------

Co-authored-by: Pam <pamhome21@gmail.com>
 2023-07-30 13:15:13 +09:00
Kohya S 4072f723c1 Merge branch 'main' into sdxl 2023-07-29 14:55:03 +09:00
Kohya S 50b53e183e re-organize import 2023-07-23 13:33:02 +09:00
青龍聖者@bdsqlsz d131bde183
Support for bitsandbytes 0.39.1 with Paged Optimizer(AdamW8bit and Lion8bit) (#631) 
* ADD libbitsandbytes.dll for 0.38.1

* Delete libbitsandbytes_cuda116.dll

* Delete cextension.py

* add main.py

* Update requirements.txt for bitsandbytes 0.38.1

* Update README.md for bitsandbytes-windows

* Update README-ja.md  for bitsandbytes 0.38.1

* Update main.py for return cuda118

* Update train_util.py for lion8bit

* Update train_README-ja.md for lion8bit

* Update train_util.py for add DAdaptAdan and DAdaptSGD

* Update train_util.py for DAdaptadam

* Update train_network.py for dadapt

* Update train_README-ja.md for DAdapt

* Update train_util.py for DAdapt

* Update train_network.py for DAdaptAdaGrad

* Update train_db.py for DAdapt

* Update fine_tune.py for DAdapt

* Update train_textual_inversion.py for DAdapt

* Update train_textual_inversion_XTI.py for DAdapt

* Revert "Merge branch 'qinglong' into main"

This reverts commit b65c023083d6d1e8a30eb42eddd603d1aac97650, reversing
changes made to f6fda20caf5e773d56bcfb5c4575c650bb85362b.

* Revert "Update requirements.txt for bitsandbytes 0.38.1"

This reverts commit 83abc60dfaddb26845f54228425b98dd67997528.

* Revert "Delete cextension.py"

This reverts commit 3ba4dfe046874393f2a022a4cbef3628ada35391.

* Revert "Update README.md for bitsandbytes-windows"

This reverts commit 4642c52086b5e9791233007e2fdfd97f832cd897.

* Revert "Update README-ja.md  for bitsandbytes 0.38.1"

This reverts commit fa6d7485ac067ebc49e6f381afdb8dd2f12caa8f.

* Update train_util.py

* Update requirements.txt

* support PagedAdamW8bit/PagedLion8bit

* Update requirements.txt

* update for PageAdamW8bit and PagedLion8bit

* Revert

* revert main
 2023-07-22 19:45:32 +09:00
Kohya S 73a08c0be0
Merge pull request #630 from ddPn08/sdxl 
make tracker init_kwargs configurable
 2023-07-20 22:05:55 +09:00
Kohya S acf16c063a make to work with PyTorch 1.12 2023-07-20 21:41:16 +09:00
Kohya S 225e871819 enable full bf16 trainint in train_network 2023-07-19 08:41:42 +09:00
Kohya S 6d2d8dfd2f add zero_terminal_snr option 2023-07-18 23:17:23 +09:00
Kohya S 0ec7166098 make crop top/left same as stabilityai's prep 2023-07-18 21:39:36 +09:00
Kohya S 41d195715d fix scheduler steps with gradient accumulation 2023-07-16 15:56:29 +09:00
Kohya S 516f64f4d9 add caching to disk for text encoder outputs 2023-07-16 14:53:47 +09:00
Kohya S 94c151aea3 refactor caching latents (flip in same npz, etc) 2023-07-15 18:28:33 +09:00
Kohya S 81fa54837f fix sampling in multi GPU training 2023-07-15 11:21:14 +09:00
Kohya S 814996b14f fix NaN in sampling image 2023-07-11 23:18:35 +09:00
ddPn08 b841dd78fe
make tracker init_kwargs configurable 2023-07-11 10:21:45 +09:00
Kohya S b6e328ea8f don't hold latent on memory for finetuning dataset 2023-07-10 08:46:15 +09:00
Kohya S 3579b4570f
Merge pull request #628 from KohakuBlueleaf/full_bf16 
Full bf16 support
 2023-07-09 14:22:44 +09:00
Kohaku-Blueleaf d974959738
Update train_util.py for full_bf16 support 2023-07-09 12:47:26 +08:00
ddPn08 d599394f60
support avif 2023-07-08 15:47:56 +09:00
Kohya S cc3d40ca44 support sdxl in prepare scipt 2023-07-07 21:16:41 +09:00
Kohya S 3060eb5baf remove debug print 2023-07-05 21:44:46 +09:00
Kohya S 2febbfe4b0 add error message for old npz 2023-07-03 20:58:35 +09:00
Kohya S ea182461d3 add min/max_timestep 2023-07-03 20:44:42 +09:00
Kohya S 64cf922841 add feature to sample images during sdxl training 2023-07-02 16:42:19 +09:00
Kohya S d395bc0647 fix max_token_length not works for sdxl 2023-06-29 13:02:19 +09:00
Kohya S 2c461e4ad3 Add no_half_vae for SDXL training, add nan check 2023-06-26 20:38:09 +09:00
Kohya S 747af145ed add sdxl fine-tuning and LoRA 2023-06-26 08:07:24 +09:00
Kohya S 9e9df2b501 update dataset to return size, refactor ctrlnet ds 2023-06-24 17:56:02 +09:00
Kohya S 92e50133f8
Merge branch 'original-u-net' into dev 2023-06-17 21:57:08 +09:00
Kohya S 19dfa24abb Merge branch 'main' into original-u-net 2023-06-16 20:59:34 +09:00
Kohya S 5845de7d7c common lr checking for dadaptation and prodigy 2023-06-15 21:47:37 +09:00
青龍聖者@bdsqlsz e97d67a681
Support for Prodigy(Dadapt variety for Dylora) (#585) 
* Update train_util.py for DAdaptLion

* Update train_README-zh.md for dadaptlion

* Update train_README-ja.md for DAdaptLion

* add DAdatpt V3

* Alignment

* Update train_util.py for experimental

* Update train_util.py V3

* Update train_README-zh.md

* Update train_README-ja.md

* Update train_util.py fix

* Update train_util.py

* support Prodigy

* add lower
 2023-06-15 21:12:53 +09:00
Kohya S 9806b00f74 add arbitrary dataset feature to each script 2023-06-15 20:39:39 +09:00
Kohya S 449ad7502c use original unet for HF models, don't download TE 2023-06-14 22:26:05 +09:00
Kohya S 9aee793078 support arbitrary dataset for train_network.py 2023-06-14 12:49:12 +09:00
ykume 9e1683cf2b support sdpa 2023-06-11 21:26:15 +09:00
ykume 0315611b11 remove workaround for accelerator=0.15, fix XTI 2023-06-11 18:32:14 +09:00
ykume 035dd3a900 fix mem_eff_attn does not work 2023-06-11 17:08:21 +09:00