1580158
/
kohya_ss
mirror of https://github.com/bmaltais/kohya_ss
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,966 Commits
16 Branches
178 Tags
68 MiB
Commit Graph

195 Commits (b9d5db8a423a4e321546eb609bfba0c572306d24)

Author SHA1 Message Date
Kohya S ec7f9bab6c
Merge branch 'dev' into dev 2023-03-19 10:25:22 +09:00
Kohya S 83e102c691 refactor config parse, feature to output config 2023-03-19 10:11:11 +09:00
Kohya S c3f9eb10f1 format with black 2023-03-18 18:58:12 +09:00
Linaqruf 44d4cfb453 feat: added function to load training config with .toml 2023-03-12 11:52:37 +07:00
Kohya S 2652c9a66c
Merge pull request #276 from mio2333/main 
Append sys path for import_module
 2023-03-10 21:43:32 +09:00
Isotr0py e3b2bb5b80
Merge branch 'dev' into dev 2023-03-10 19:04:07 +08:00
Isotr0py 7544b38635 fix multi gpu 2023-03-10 18:45:53 +08:00
mio 68cd874bb6
Append sys path for import_module 
This will be better if we run the scripts we do not run the training script from the current directory.  This is reasonable as some other projects will use this as a subfolder, such as https://github.com/ddPn08/kohya-sd-scripts-webui. I can not run the script without adding this.
 2023-03-10 18:29:34 +08:00
Kohya S 458173da5e
Merge branch 'dev' into dev 2023-03-10 13:00:49 +09:00
Isotr0py eb68892ab1 add lr_scheduler_type etc 2023-03-09 16:51:22 +08:00
Kohya S 3ce846525b set minimum metadata even with no_metadata 2023-03-08 21:19:12 +09:00
ddPn08 87846c043f
fix for multi gpu training 2023-03-08 09:46:37 +09:00
Kohya S 5bb571ccc0 Merge branch 'main' into dev 2023-03-06 17:49:47 +09:00
camenduru 772ee52ef2
metadata |= to metadata.update 2023-03-06 00:31:28 +03:00
Isotr0py 4396350271 Add network_args to meta if using another network 2023-03-04 13:59:22 +08:00
Kohya S 5602e0e5fc change dataset config option to dataset_config 2023-03-02 21:51:58 +09:00
Kohya S 09f575fd4d merge image_dir for metadata editor 2023-03-02 21:17:25 +09:00
Kohya S 7e1aa5f4d6 keep tag_frequency for each dataset 2023-03-02 20:27:22 +09:00
Kohya S e9f37c4049 do not save image_dir to metadata if None 2023-03-01 23:37:17 +09:00
Kohya S c95943b663 merge tag frequence for metadata editor 2023-03-01 22:10:43 +09:00
Kohya S ed19a92bbe fix typos 2023-03-01 21:01:10 +09:00
fur0ut0 8abb8645ae
add detail dataset config feature by extra config file (#227) 
* add config file schema

* change config file specification

* refactor config utility

* unify batch_size to train_batch_size

* fix indent size

* use batch_size instead of train_batch_size

* make cache_latents configurable on subset

* rename options
* bucket_repo_range
* shuffle_keep_tokens

* update readme

* revert to min_bucket_reso & max_bucket_reso

* use subset structure in dataset

* format import lines

* split mode specific options

* use only valid subset

* change valid subsets name

* manage multiple datasets by dataset group

* update config file sanitizer

* prune redundant validation

* add comments

* update type annotation

* rename json_file_name to metadata_file

* ignore when image dir is invalid

* fix tag shuffle and dropout

* ignore duplicated subset

* add method to check latent cachability

* fix format

* fix bug

* update caption dropout default values

* update annotation

* fix bug

* add option to enable bucket shuffle across dataset

* update blueprint generate function

* use blueprint generator for dataset initialization

* delete duplicated function

* update config readme

* delete debug print

* print dataset and subset info as info

* enable bucket_shuffle_across_dataset option

* update config readme for clarification

* compensate quotes for string option example

* fix bug of bad usage of join

* conserve trained metadata backward compatibility

* enable shuffle in data loader by default

* delete resolved TODO

* add comment for image data handling

* fix reference bug

* fix undefined variable bug

* prevent raise overwriting

* assert image_dir and metadata_file validity

* add debug message for ignoring subset

* fix inconsistent import statement

* loosen too strict validation on float value

* sanitize argument parser separately

* make image_dir optional for fine tuning dataset

* fix import

* fix trailing characters in print

* parse flexible dataset config deterministically

* use relative import

* print supplementary message for parsing error

* add note about different methods

* add note of benefit of separate dataset

* add error example

* add note for english readme plan

---------

Co-authored-by: Kohya S <52813779+kohya-ss@users.noreply.github.com>
 2023-03-01 20:58:08 +09:00
Kohya S dd523c94ff sample images in training (not fully tested) 2023-02-27 17:48:32 +09:00
Kohya S 0eb01dea55 add max_grad_norm to metadata 2023-02-23 21:34:38 +09:00
Kohya S 95b5aed41b
Merge pull request #221 from space-nuko/add-more-metadata 
Add more missing metadata
 2023-02-23 21:14:26 +09:00
Kohya S f403ac6132 fix float32 training doesn't work in some case 2023-02-23 20:56:41 +09:00
space-nuko b39cf6e2c0 Add more missing metadata 2023-02-23 02:25:24 -08:00
Kohya S 9ab964d0b8 Add Adafactor optimzier 2023-02-22 21:09:47 +09:00
Kohya S 663aad2b0d refactor get_scheduler etc. 2023-02-20 22:47:43 +09:00
Kohya S 107fa754e5
Merge branch 'dev' into optimizer-expand-and-refactor 2023-02-20 20:12:42 +09:00
mgz-dev b29c5a750c expand optimizer options and refactor 
Refactor code to make it easier to add new optimizers, and support alternate optimizer parameters

-move redundant code to train_util for initializing optimizers
- add SGD Nesterov optimizers as option (since they are already available)
- add new parameters which may be helpful for tuning existing and new optimizers
 2023-02-19 17:45:09 -06:00
unknown 045a3dbe48 apply dadaptation 2023-02-19 18:37:07 +09:00
space-nuko 4e5db58a71 Add optimizer to metadata 2023-02-18 23:28:36 -08:00
Kohya S 048e7cd428 add lion optimizer support 2023-02-19 15:26:14 +09:00
Isotr0py 78d1fb5ce6 Add '--lowram' argument 2023-02-17 12:08:54 +08:00
Kohya S d01d953262
Merge pull request #196 from space-nuko/add-noise-offset-metadata 
Add noise offset to metadata
 2023-02-16 22:01:02 +09:00
Kohya S 914d1505df
Merge pull request #189 from shirayu/improve_loss_track 
Show the moving average loss
 2023-02-16 22:00:26 +09:00
space-nuko 496c8cdc09 Add noise-offset to metadata 2023-02-16 02:56:39 -08:00
Kohya S 43c0a69843 Add noise_offset 2023-02-14 21:15:48 +09:00
Yuta Hayashibe 8aed5125de Removed call of sum() 2023-02-14 21:11:30 +09:00
Kohya S e0f007f2a9 Fix import 2023-02-14 20:55:38 +09:00
Kohya S 8f1e930bf4
Merge pull request #187 from space-nuko/add-commit-hash 
Add commit hash to metadata
 2023-02-14 19:52:30 +09:00
Yuta Hayashibe 21f5b618c3 Show the moving average loss 2023-02-14 19:46:27 +09:00
space-nuko 5471b0deb0 Add commit hash to metadata 2023-02-13 02:58:06 -08:00
Isotr0py 2b1a3080e7 Add type checking 2023-02-12 15:32:38 +08:00
Isotr0py 92a1af8024
Merge branch 'kohya-ss:main' into support-multi-gpu 2023-02-12 15:06:46 +08:00
Kohya S 3a72e6f003 add tag dropout 2023-02-09 21:35:27 +09:00
Isotr0py b8ad17902f
fix get_hidden_states expected scalar Error again 2023-02-08 23:09:59 +08:00
Isotr0py 9a9ac79edf correct wrong inserted code for noise_pred fix 2023-02-08 22:30:20 +08:00
Isotr0py 6473aa1dd7 fix Input type error in noise_pred when using DDP 2023-02-08 21:32:21 +08:00
Isotr0py b599adc938 fix Input type error when using DDP 2023-02-08 20:14:20 +08:00
Isotr0py fb312acb7f support DistributedDataParallel 2023-02-08 20:12:43 +08:00
Isotr0py 938bd71844 lower ram usage 2023-02-08 18:31:27 +08:00
Kohya S e42b2f7aa9 conditional caption dropout (in progress) 2023-02-07 22:28:56 +09:00
forestsource 7db98baa86 Add dropout options 2023-02-07 00:01:30 +09:00
Kohya S ea2dfd09ef update bucketing features 2023-02-05 21:37:46 +09:00
Kohya S 9fd7fb813d
Merge branch 'dev' into main 2023-02-04 18:16:03 +09:00
Kohya S 58a809eaff Add comment 2023-02-03 21:04:03 +09:00
hitomi 26a81d075c add --persistent_data_loader_workers option 2023-02-01 16:02:15 +08:00
Kohya S 74eba06d13
Merge pull request #104 from space-nuko/caption-frequency-metadata 
Add tag frequency metadata
 2023-01-31 20:56:15 +09:00
Kohya S 7817e95a86 change name of arg 2023-01-29 20:28:24 +09:00
Kohya S ed2e431950
Merge branch 'main' into caption-frequency-metadata 2023-01-29 17:50:23 +09:00
michaelgzhang 0fef7b4684 monkeypatch updated get_scheduler for diffusers 
enables use of "num_cycles" and "power" for cosine_with_restarts and polynomial learning rate schedulers
 2023-01-27 16:42:11 -06:00
Kohya S 9f644d8dc3 Change default save format to safetensors 2023-01-24 20:16:21 +09:00
Kohya S 36dc97c841
Merge pull request #103 from space-nuko/bucketing-metadata 
Add bucketing metadata
 2023-01-24 19:06:21 +09:00
Kohya S 7f17237ada
Merge pull request #92 from forestsource/add_save_n_epoch_ratio 
Add save_n_epoch_ratio
 2023-01-24 18:59:47 +09:00
space-nuko 2e8a3d20dd Add tag frequency metadata 2023-01-23 17:43:03 -08:00
space-nuko 66051883fb Add bucketing metadata 2023-01-23 17:26:58 -08:00
Kohya S a7218574f2 Update help message 2023-01-22 21:33:48 +09:00
Kohya S 8746188ed7 Add traning_comment metadata. 2023-01-22 18:33:19 +09:00
forestsource 5e817e4343 Add save_n_epoch_ratio 2023-01-22 03:00:28 +09:00
Kohya S b4636d4185 Add scaling alpha for LoRA 2023-01-21 20:37:34 +09:00
Kohya S 22ee0ac467 Move TE/UN loss calc to train script 2023-01-21 12:51:17 +09:00
Kohya S 17089b1287 Merge branch 'dev' of https://github.com/kohya-ss/sd-scripts into dev 2023-01-21 12:46:20 +09:00
Kohya S 7ee808d5d7
Merge pull request #79 from mgz-dev/tensorboard-improvements 
expand details in tensorboard logs
 2023-01-21 12:46:13 +09:00
Kohya S 9ff26af68b Update to add grad_ckpting etc to metadata 2023-01-21 12:36:31 +09:00
Kohya S 7dbcef745a
Merge pull request #77 from space-nuko/ss-extra-metadata 
More helpful metadata
 2023-01-21 12:18:23 +09:00
Kohya S 758323532b add save_last_n_epochs_state to train_network 2023-01-19 20:59:45 +09:00
Kohya S e6a8c9d269 Fix some LoRA not trained if gradient checkpointing 2023-01-19 20:39:33 +09:00
space-nuko da48f74e7b Add new version model/VAE hash to training metadata 2023-01-18 23:00:16 -08:00
michaelgzhang 303c3410e2 expand details in tensorboard logs 
- Update tensorboard logging to track both unet and textencoder learning rates
- Update tensorboard logging to track both current and moving average epoch loss
- Clean up tensorboard log variable names for dashboard formatting
 2023-01-18 13:10:13 -06:00
space-nuko de1dde1a06 More helpful metadata 
- dataset/reg image dirs
- random session ID
- keep_tokens
- training date
- output name
 2023-01-17 16:28:35 -08:00
Kohya S aa40cb9345 Add train epochs and max workers option to train 2023-01-15 13:07:47 +09:00
Kohya S e4f9b2b715 Add VAE to meatada, add no_metadata option 2023-01-11 23:12:18 +09:00
space-nuko 0c4423d9dc Add epoch number to metadata 2023-01-10 02:50:04 -08:00
space-nuko 2e4ce0fdff Add training metadata to output LoRA model 2023-01-10 02:49:52 -08:00
Kohya S fbaf373c8a fix gradient accum not used for lr schduler 2023-01-09 13:13:37 +09:00
Kohya S 6b62c44022 fix errors in fine tuning 2023-01-08 21:40:40 +09:00
Kohya S 9f1d3aca24 add save_state_on_train end, fix reg imgs repeats 2023-01-07 20:20:37 +09:00
Kohya S f56988b252 unify dataset and save functions 2023-01-05 08:10:22 +09:00
Kohya S 4c35006731 split common function from train_network to util 2023-01-03 20:22:25 +09:00
Kohya S 6b522b34c1 move code for xformers to train_util 2023-01-02 16:08:21 +09:00
Yuta Hayashibe 0f20453997 Fix typos 2022-12-31 14:39:36 +09:00
Kohya S 64de791f2c set default dataset_repeats to 1, resolution check 2022-12-29 20:12:09 +09:00
Kohya S 445b34de1f Add LoRA training/generating. 2022-12-25 21:34:59 +09:00