1580158
/
kohya_ss
mirror of https://github.com/bmaltais/kohya_ss
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Adding support for more config.toml values

pull/2219/head
bmaltais 2024-04-03 22:19:57 -04:00
parent ef1dcafbfe
commit 0aa91418b3
6 changed files with 170 additions and 162 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

72
README.md
View File

@ -51,16 +51,6 @@ The GUI allows you to set the training parameters and generate and run the requi
- [Software Updates](#software-updates)
- [Recommendations for Users](#recommendations-for-users)
- [2024/03/13 (v23.0.11)](#20240313-v23011)
- [2024/03/13 (v23.0.9)](#20240313-v2309)
- [2024/03/12 (v23.0.8)](#20240312-v2308)
- [2024/03/12 (v23.0.7)](#20240312-v2307)
- [2024/03/11 (v23.0.6)](#20240311-v2306)
- [2024/03/11 (v23.0.5)](#20240311-v2305)
- [2024/03/10 (v23.0.4)](#20240310-v2304)
- [2024/03/10 (v23.0.3)](#20240310-v2303)
- [2024/03/10 (v23.0.2)](#20240310-v2302)
- [2024/03/09 (v23.0.1)](#20240309-v2301)
- [2024/03/02 (v23.0.0)](#20240302-v2300)
## 🦒 Colab
@ -345,6 +335,8 @@ To use the default configuration file, follow these steps:
This approach allows you to easily adjust the configuration to suit your specific needs to open the desired default folders for each type of folder/file input supported in the GUI.
You can specify the path to your config.toml (or any other name you like) when running the GUI. For instance: ./gui.bat --config c:\my_config.toml
## LoRA
To train a LoRA, you can currently use the `train_network.py` code. You can create a LoRA network by using the all-in-one GUI.
@ -387,10 +379,13 @@ If you encounter an X error related to the page file, you may need to increase t
If you encounter an error indicating that the module `tkinter` is not found, try reinstalling Python 3.10 on your system.
### LORA Training on TESLA V100 - GPU Utilization Issue
#### Issue Summary
When training LORA on a TESLA V100, users reported low GPU utilization. Additionally, there was difficulty in specifying GPUs other than the default for training.
#### Potential Solutions
- **GPU Selection:** Users can specify GPU IDs in the setup configuration to select the desired GPUs for training.
- **Improving GPU Load:** Utilizing `adamW8bit` optimizer and increasing the batch size can help achieve 70-80% GPU utilization without exceeding GPU memory limits.
@ -436,11 +431,11 @@ ControlNet dataset is used to specify the mask. The mask images should be the RG
- The options `--sample_every_n_epochs` and `--sample_every_n_steps` in each training script now display a warning and ignore them when a number less than or equal to `0` is specified. Thanks to S-Del for raising the issue.
- The [English version of the dataset settings documentation](./docs/config_README-en.md) is added. PR [#1175](https://github.com/kohya-ss/sd-scripts/pull/1175) Thanks to darkstorm2150!
- Add GUI support for the new parameters listed above.
- Move accelerate launch parameters to new `Accelerate launch` accordion above `Model` accordion.
- Add support for `Debiased Estimation loss` to Dreambooth settings.
- Add support for "Dataset Preparation" defaults via the config.toml file.
- Add field to allow for the input of extra accelerate launch arguments.
- Added GUI support for the new parameters listed above.
- Moved accelerate launch parameters to a new `Accelerate launch` accordion above the `Model` accordion.
- Added support for `Debiased Estimation loss` to Dreambooth settings.
- Added support for "Dataset Preparation" defaults via the config.toml file.
- Added a field to allow for the input of extra accelerate launch arguments.
### 2024/03/21 (v23.0.15)
@ -491,50 +486,3 @@ ControlNet dataset is used to specify the mask. The mask images should be the RG
- Increase icon size.
- More setup fixes.
### 2024/03/13 (v23.0.9)
- Reworked how setup can be run to improve Stability Matrix support.
- Added support for huggingface-based vea path.
### 2024/03/12 (v23.0.8)
- Add the ability to create output and logs folder if it does not exist
### 2024/03/12 (v23.0.7)
- Fixed minor issues related to functions and file paths.
### 2024/03/11 (v23.0.6)
- Fixed an issue with PYTHON paths that have "spaces" in them.
### 2024/03/11 (v23.0.5)
- Updated python module verification.
- Removed cudnn module installation in Windows.
### 2024/03/10 (v23.0.4)
- Updated bitsandbytes to 0.43.0.
- Added packaging to runpod setup.
### 2024/03/10 (v23.0.3)
- Fixed a bug with setup.
- Enforced proper python version before running the GUI to prevent issues with execution of the GUI.
### 2024/03/10 (v23.0.2)
- Improved validation of the path provided by users before running training.
### 2024/03/09 (v23.0.1)
- Updated bitsandbytes module to 0.43.0 as it provides native Windows support.
- Minor fixes to the code.
### 2024/03/02 (v23.0.0)
- Used sd-scripts release [0.8.4](https://github.com/kohya-ss/sd-scripts/releases/tag/v0.8.4) post commit [fccbee27277d65a8dcbdeeb81787ed4116b92e0b](https://github.com/kohya-ss/sd-scripts/commit/fccbee27277d65a8dcbdeeb81787ed4116b92e0b).
- Major code refactoring thanks to @wkpark. This will make updating sd-scripts cleaner by keeping sd-scripts files separate from the GUI files. This will also make configuration more streamlined with fewer tabs and more accordion elements. Hope you like the new style.
- This new release is implementing a significant structure change, moving all of the sd-scripts written by kohya under a folder called sd-scripts in the root of this project. This folder is a submodule that will be populated during setup or GUI execution.

80
config example.toml
View File

@ -2,23 +2,81 @@
# Edit the values to suit your needs
# Default folders location
[model]
models_dir = "./models" # Pretrained model name or path
output_name = "new model" # Trained model output name
train_data_dir = "./data" # Image folder (containing training images subfolders) / Image folder (containing training images)
dataset_config = "./test.toml" # Dataset config file (Optional. Select the toml configuration file to use for the dataset)
training_comment = "Some training comment" # Training comment
save_model_as = "safetensors" # Save model as (ckpt, safetensors, diffusers, diffusers_safetensors)
save_precision = "bf16" # Save model precision (fp16, bf16, float)
[folders]
output_dir = "./outputs" # Output directory for trained model
reg_data_dir = "./data/reg" # Regularisation directory
logging_dir = "./logs" # Logging directory
config_dir = "./presets" # Load/Save Config file
log_tracker_config_dir = "./logs" # Log tracker configs directory
state_dir = "./outputs" # Resume from saved training state
vae_dir = "./models/vae" # VAEs folder path
# Example custom folder location
# models_dir = "e:/models" # Pretrained model name or path
[configuration]
config_dir = "./presets" # Load/Save Config file
[advanced]
adaptive_noise_scale = 0 # Adaptive noise scale
additional_parameters = "" # Additional parameters
bucket_no_upscale = true # Don't upscale bucket resolution
bucket_reso_steps = 64 # Bucket resolution steps
caption_dropout_every_n_epochs = 0 # Caption dropout every n epochs
caption_dropout_rate = 0 # Caption dropout rate
color_aug = false # Color augmentation
clip_skip = 1 # Clip skip
debiased_estimation_loss = false # Debiased estimation loss
flip_aug = false # Flip augmentation
fp8_base = false # FP8 base training (experimental)
full_bf16 = false # Full bf16 training (experimental)
full_fp16 = false # Full fp16 training (experimental)
gradient_accumulation_steps = 1 # Gradient accumulation steps
gradient_checkpointing = false # Gradient checkpointing
ip_noise_gamma = 0 # IP noise gamma
ip_noise_gamma_random_strength = false # IP noise gamma random strength (true, false)
keep_tokens = 0 # Keep tokens
log_tracker_config_dir = "./logs" # Log tracker configs directory
log_tracker_name = "" # Log tracker name
masked_loss = false # Masked loss
max_data_loader_n_workers = "0" # Max data loader n workers (string)
max_timestep = 1000 # Max timestep
max_token_length = "150" # Max token length ("75", "150", "225")
mem_eff_attn = false # Memory efficient attention
min_snr_gamma = 0 # Min SNR gamma
min_timestep = 0 # Min timestep
multires_noise_iterations = 0 # Multires noise iterations
multires_noise_discount = 0 # Multires noise discount
no_token_padding = false # Disable token padding
noise_offset = 0 # Noise offset
noise_offset_random_strength = false # Noise offset random strength (true, false)
noise_offset_type = "Original" # Noise offset type ("Original", "Multires")
persistent_data_loader_workers = false # Persistent data loader workers
prior_loss_weight = 1.0 # Prior loss weight
random_crop = false # Random crop
save_every_n_steps = 0 # Save every n steps
save_last_n_steps = 0 # Save last n steps
save_last_n_steps_state = 0 # Save last n steps state
save_state = false # Save state
save_state_on_train_end = false # Save state on train end
scale_v_pred_loss_like_noise_pred = false # Scale v pred loss like noise pred
shuffle_caption = false # Shuffle captions
state_dir = "./outputs" # Resume from saved training state
use_wandb = false # Use wandb
vae_batch_size = 0 # VAE batch size
vae_dir = "./models/vae" # VAEs folder path
v_pred_like_loss = 0 # V pred like loss weight
wandb_api_key = "" # Wandb api key
wandb_run_name = "" # Wandb run name
weighted_captions = false # Weighted captions
xformers = "xformers" # CrossAttention (none, sdp, xformers)
# This next section can be used to set default values for the Dataset Preparation section
# The "Destination training direcroty" field will be equal to "train_data_dir" as specified above
# [dataset_preparation]
# instance_prompt = "instance"
# class_prompt = "class"
# images_folder = "/some/folder/where/images/are"
# reg_images_folder = "/some/folder/where/reg/images/are"
[dataset_preparation]
instance_prompt = "instance"
class_prompt = "class"
images_folder = "/some/folder/where/images/are"
reg_images_folder = "/some/folder/where/reg/images/are"

127
kohya_gui/class_advanced_training.py
View File

@ -46,10 +46,10 @@ class AdvancedTraining:
self.config = config
# Determine the current directories for VAE and output, falling back to defaults if not specified.
self.current_vae_dir = self.config.get("vae_dir", "./models/vae")
self.current_state_dir = self.config.get("state_dir", "./outputs")
self.current_vae_dir = self.config.get("advanced.vae_dir", "./models/vae")
self.current_state_dir = self.config.get("advanced.state_dir", "./outputs")
self.current_log_tracker_config_dir = self.config.get(
"log_tracker_config_dir", "./logs"
"advanced.log_tracker_config_dir", "./logs"
)
# Define the behavior for changing noise offset type.
@ -75,19 +75,19 @@ class AdvancedTraining:
# Exclude token padding option for LoRA training type.
if training_type != "lora":
self.no_token_padding = gr.Checkbox(
label="No token padding", value=False
label="No token padding", value=self.config.get("advanced.no_token_padding", False)
)
self.gradient_accumulation_steps = gr.Slider(
label="Gradient accumulate steps",
info="Number of updates steps to accumulate before performing a backward/update pass",
value="1",
value=self.config.get("advanced.gradient_accumulation_steps", 1),
minimum=1,
maximum=120,
step=1,
)
self.weighted_captions = gr.Checkbox(label="Weighted captions", value=False)
self.weighted_captions = gr.Checkbox(label="Weighted captions", value=self.config.get("advanced.weighted_captions", False))
with gr.Group(), gr.Row(visible=not finetuning):
self.prior_loss_weight = gr.Number(label="Prior loss weight", value=1.0)
self.prior_loss_weight = gr.Number(label="Prior loss weight", value=self.config.get("advanced.prior_loss_weight", 1.0))
def list_vae_files(path):
self.current_vae_dir = path if not path == "" else "."
@ -96,14 +96,14 @@ class AdvancedTraining:
self.vae = gr.Dropdown(
label="VAE (Optional: Path to checkpoint of vae for training)",
interactive=True,
choices=[""] + list_vae_files(self.current_vae_dir),
value="",
choices=[self.config.get("advanced.vae_dir", "")] + list_vae_files(self.current_vae_dir),
value=self.config.get("advanced.vae_dir", ""),
allow_custom_value=True,
)
create_refresh_button(
self.vae,
lambda: None,
lambda: {"choices": [""] + list_vae_files(self.current_vae_dir)},
lambda: {"choices": [self.config.get("advanced.vae_dir", "")] + list_vae_files(self.current_vae_dir)},
"open_folder_small",
)
self.vae_button = gr.Button(
@ -116,7 +116,7 @@ class AdvancedTraining:
)
self.vae.change(
fn=lambda path: gr.Dropdown(choices=[""] + list_vae_files(path)),
fn=lambda path: gr.Dropdown(choices=[self.config.get("advanced.vae_dir", "")] + list_vae_files(path)),
inputs=self.vae,
outputs=self.vae,
show_progress=False,
@ -126,23 +126,24 @@ class AdvancedTraining:
self.additional_parameters = gr.Textbox(
label="Additional parameters",
placeholder='(Optional) Use to provide additional parameters not handled by the GUI. Eg: --some_parameters "value"',
value=self.config.get("advanced.additional_parameters", ""),
)
with gr.Row():
self.save_every_n_steps = gr.Number(
label="Save every N steps",
value=0,
value=self.config.get("advanced.save_every_n_steps", 0),
precision=0,
info="(Optional) The model is saved every specified steps",
)
self.save_last_n_steps = gr.Number(
label="Save last N steps",
value=0,
value=self.config.get("advanced.save_last_n_steps", 0),
precision=0,
info="(Optional) Save only the specified number of models (old models will be deleted)",
)
self.save_last_n_steps_state = gr.Number(
label="Save last N steps state",
value=0,
value=self.config.get("advanced.save_last_n_steps_state", 0),
precision=0,
info="(Optional) Save only the specified number of states (old models will be deleted)",
)
@ -161,10 +162,10 @@ class AdvancedTraining:
), gr.Checkbox(interactive=full_bf16_active)
self.keep_tokens = gr.Slider(
label="Keep n tokens", value="0", minimum=0, maximum=32, step=1
label="Keep n tokens", value=self.config.get("advanced.keep_tokens", 0), minimum=0, maximum=32, step=1
)
self.clip_skip = gr.Slider(
label="Clip skip", value="1", minimum=1, maximum=12, step=1
label="Clip skip", value=self.config.get("advanced.clip_skip", 1), minimum=1, maximum=12, step=1
)
self.max_token_length = gr.Dropdown(
label="Max Token Length",
@ -173,7 +174,7 @@ class AdvancedTraining:
"150",
"225",
],
value="75",
value=self.config.get("advanced.max_token_length", "75"),
)
with gr.Row():
@ -181,15 +182,15 @@ class AdvancedTraining:
self.fp8_base = gr.Checkbox(
label="fp8 base training (experimental)",
info="U-Net and Text Encoder can be trained with fp8 (experimental)",
value=False,
value=self.config.get("advanced.fp8_base", False),
)
self.full_fp16 = gr.Checkbox(
label="Full fp16 training (experimental)",
value=False,
value=self.config.get("advanced.full_fp16", False),
)
self.full_bf16 = gr.Checkbox(
label="Full bf16 training (experimental)",
value=False,
value=self.config.get("advanced.full_bf16", False),
info="Required bitsandbytes >= 0.36.0",
)
@ -206,45 +207,45 @@ class AdvancedTraining:
with gr.Row():
self.gradient_checkpointing = gr.Checkbox(
label="Gradient checkpointing", value=False
label="Gradient checkpointing", value=self.config.get("advanced.gradient_checkpointing", False)
)
self.shuffle_caption = gr.Checkbox(label="Shuffle caption", value=False)
self.shuffle_caption = gr.Checkbox(label="Shuffle caption", value=self.config.get("advanced.shuffle_caption", False))
self.persistent_data_loader_workers = gr.Checkbox(
label="Persistent data loader", value=False
label="Persistent data loader", value=self.config.get("advanced.persistent_data_loader_workers", False)
)
self.mem_eff_attn = gr.Checkbox(
label="Memory efficient attention", value=False
label="Memory efficient attention", value=self.config.get("advanced.mem_eff_attn", False)
)
with gr.Row():
self.xformers = gr.Dropdown(
label="CrossAttention",
choices=["none", "sdpa", "xformers"],
value="xformers",
value=self.config.get("advanced.xformers", "xformers"),
)
self.color_aug = gr.Checkbox(
label="Color augmentation",
value=False,
value=self.config.get("advanced.color_aug", False),
info="Enable weak color augmentation",
)
self.flip_aug = gr.Checkbox(
label="Flip augmentation",
value=False,
value=getattr(self.config, "advanced.flip_aug", False),
info="Enable horizontal flip augmentation",
)
self.masked_loss = gr.Checkbox(
label="Masked loss",
value=False,
value=self.config.get("advanced.masked_loss", False),
info="Apply mask for calculating loss. conditioning_data_dir is required for dataset",
)
with gr.Row():
self.scale_v_pred_loss_like_noise_pred = gr.Checkbox(
label="Scale v prediction loss",
value=False,
value=self.config.get("advanced.scale_v_pred_loss_like_noise_pred", False),
info="Only for SD v2 models. By scaling the loss according to the time step, the weights of global noise prediction and local noise prediction become the same, and the improvement of details may be expected.",
)
self.min_snr_gamma = gr.Slider(
label="Min SNR gamma",
value=0,
value=self.config.get("advanced.min_snr_gamma", 0),
minimum=0,
maximum=20,
step=1,
@ -252,26 +253,26 @@ class AdvancedTraining:
)
self.debiased_estimation_loss = gr.Checkbox(
label="Debiased Estimation loss",
value=False,
value=self.config.get("advanced.debiased_estimation_loss", False),
info="Automates the processing of noise, allowing for faster model fitting, as well as balancing out color issues. Do not use if Min SNR gamma is specified.",
)
with gr.Row():
# self.sdpa = gr.Checkbox(label='Use sdpa', value=False, info='Use sdpa for CrossAttention')
self.bucket_no_upscale = gr.Checkbox(
label="Don't upscale bucket resolution", value=True
label="Don't upscale bucket resolution", value=self.config.get("advanced.bucket_no_upscale", True)
)
self.bucket_reso_steps = gr.Slider(
label="Bucket resolution steps",
value=64,
value=self.config.get("advanced.bucket_reso_steps", 64),
minimum=1,
maximum=128,
)
self.random_crop = gr.Checkbox(
label="Random crop instead of center crop", value=False
label="Random crop instead of center crop", value=self.config.get("advanced.random_crop", False)
)
self.v_pred_like_loss = gr.Slider(
label="V Pred like loss",
value=0,
value=self.config.get("advanced.v_pred_like_loss", 0),
minimum=0,
maximum=1,
step=0.01,
@ -281,7 +282,7 @@ class AdvancedTraining:
with gr.Row():
self.min_timestep = gr.Slider(
label="Min Timestep",
value=0,
value=self.config.get("advanced.min_timestep", 0),
step=1,
minimum=0,
maximum=1000,
@ -289,7 +290,7 @@ class AdvancedTraining:
)
self.max_timestep = gr.Slider(
label="Max Timestep",
value=1000,
value=self.config.get("advanced.max_timestep", 1000),
step=1,
minimum=0,
maximum=1000,
@ -303,13 +304,13 @@ class AdvancedTraining:
"Original",
"Multires",
],
value="Original",
value=self.config.get("advanced.noise_offset_type", "Original"),
scale=1,
)
with gr.Row(visible=True) as self.noise_offset_original:
self.noise_offset = gr.Slider(
label="Noise offset",
value=0,
value=self.config.get("advanced.noise_offset", 0),
minimum=0,
maximum=1,
step=0.01,
@ -317,12 +318,12 @@ class AdvancedTraining:
)
self.noise_offset_random_strength = gr.Checkbox(
label="Noise offset random strength",
value=False,
value=self.config.get("advanced.noise_offset_random_strength", False),
info="Use random strength between 0~noise_offset for noise offset",
)
self.adaptive_noise_scale = gr.Slider(
label="Adaptive noise scale",
value=0,
value=self.config.get("advanced.adaptive_noise_scale", 0),
minimum=-1,
maximum=1,
step=0.001,
@ -331,7 +332,7 @@ class AdvancedTraining:
with gr.Row(visible=False) as self.noise_offset_multires:
self.multires_noise_iterations = gr.Slider(
label="Multires noise iterations",
value=0,
value=self.config.get("advanced.multires_noise_iterations", 0),
minimum=0,
maximum=64,
step=1,
@ -339,7 +340,7 @@ class AdvancedTraining:
)
self.multires_noise_discount = gr.Slider(
label="Multires noise discount",
value=0,
value=self.config.get("advanced.multires_noise_discount", 0),
minimum=0,
maximum=1,
step=0.01,
@ -348,7 +349,7 @@ class AdvancedTraining:
with gr.Row(visible=True):
self.ip_noise_gamma = gr.Slider(
label="IP noise gamma",
value=0,
value=self.config.get("advanced.ip_noise_gamma", 0),
minimum=0,
maximum=1,
step=0.01,
@ -356,7 +357,7 @@ class AdvancedTraining:
)
self.ip_noise_gamma_random_strength = gr.Checkbox(
label="IP noise gamma random strength",
value=False,
value=self.config.get("advanced.ip_noise_gamma_random_strength", False),
info="Use random strength between 0~ip_noise_gamma for input perturbation noise",
)
self.noise_offset_type.change(
@ -369,19 +370,19 @@ class AdvancedTraining:
)
with gr.Row():
self.caption_dropout_every_n_epochs = gr.Number(
label="Dropout caption every n epochs", value=0
label="Dropout caption every n epochs", value=self.config.get("advanced.caption_dropout_every_n_epochs", 0),
)
self.caption_dropout_rate = gr.Slider(
label="Rate of caption dropout", value=0, minimum=0, maximum=1
label="Rate of caption dropout", value=self.config.get("advanced.caption_dropout_rate", 0), minimum=0, maximum=1
)
self.vae_batch_size = gr.Slider(
label="VAE batch size", minimum=0, maximum=32, value=0, step=1
label="VAE batch size", minimum=0, maximum=32, value=self.config.get("advanced.vae_batch_size", 0), step=1
)
with gr.Group(), gr.Row():
self.save_state = gr.Checkbox(label="Save training state", value=False)
self.save_state = gr.Checkbox(label="Save training state", value=self.config.get("advanced.save_state", False))
self.save_state_on_train_end = gr.Checkbox(
label="Save training state at end of training", value=False
label="Save training state at end of training", value=self.config.get("advanced.save_state_on_train_end", False)
)
def list_state_dirs(path):
@ -390,15 +391,15 @@ class AdvancedTraining:
self.resume = gr.Dropdown(
label='Resume from saved training state (path to "last-state" state folder)',
choices=[""] + list_state_dirs(self.current_state_dir),
value="",
choices=[self.config.get("advanced.state_dir", "")] + list_state_dirs(self.current_state_dir),
value=self.config.get("advanced.state_dir", ""),
interactive=True,
allow_custom_value=True,
)
create_refresh_button(
self.resume,
lambda: None,
lambda: {"choices": [""] + list_state_dirs(self.current_state_dir)},
lambda: {"choices": [self.config.get("advanced.state_dir", "")] + list_state_dirs(self.current_state_dir)},
"open_folder_small",
)
self.resume_button = gr.Button(
@ -410,7 +411,7 @@ class AdvancedTraining:
show_progress=False,
)
self.resume.change(
fn=lambda path: gr.Dropdown(choices=[""] + list_state_dirs(path)),
fn=lambda path: gr.Dropdown(choices=[self.config.get("advanced.state_dir", "")] + list_state_dirs(path)),
inputs=self.resume,
outputs=self.resume,
show_progress=False,
@ -418,23 +419,23 @@ class AdvancedTraining:
self.max_data_loader_n_workers = gr.Textbox(
label="Max num workers for DataLoader",
placeholder="(Optional) Override number of epoch. Default: 8",
value="0",
value=self.config.get("advanced.max_data_loader_n_workers", "0"),
)
with gr.Row():
self.use_wandb = gr.Checkbox(
label="WANDB Logging",
value=False,
value=self.config.get("advanced.use_wandb", False),
info="If unchecked, tensorboard will be used as the default for logging.",
)
self.wandb_api_key = gr.Textbox(
label="WANDB API Key",
value="",
value=self.config.get("advanced.wandb_api_key", ""),
placeholder="(Optional)",
info="Users can obtain and/or generate an api key in the their user settings on the website: https://wandb.ai/login",
)
self.wandb_run_name = gr.Textbox(
label="WANDB run name",
value="",
value=self.config.get("advanced.wandb_run_name", ""),
placeholder="(Optional)",
info="The name of the specific wandb session",
)
@ -446,15 +447,15 @@ class AdvancedTraining:
self.log_tracker_name = gr.Textbox(
label="Log tracker name",
value="",
value=self.config.get("advanced.log_tracker_name", ""),
placeholder="(Optional)",
info="Name of tracker to use for logging, default is script-specific default name",
)
self.log_tracker_config = gr.Dropdown(
label="Log tracker config",
choices=[""]
choices=[self.config.get("log_tracker_config_dir", "")]
+ list_log_tracker_config_files(self.current_log_tracker_config_dir),
value="",
value=self.config.get("log_tracker_config_dir", ""),
info="Path to tracker config file to use for logging",
interactive=True,
allow_custom_value=True,
@ -463,7 +464,7 @@ class AdvancedTraining:
self.log_tracker_config,
lambda: None,
lambda: {
"choices": [""]
"choices": [self.config.get("log_tracker_config_dir", "")]
+ list_log_tracker_config_files(self.current_log_tracker_config_dir)
},
"open_folder_small",
@ -478,7 +479,7 @@ class AdvancedTraining:
)
self.log_tracker_config.change(
fn=lambda path: gr.Dropdown(
choices=[""] + list_log_tracker_config_files(path)
choices=[self.config.get("log_tracker_config_dir", "")] + list_log_tracker_config_files(path)
),
inputs=self.log_tracker_config,
outputs=self.log_tracker_config,

4
kohya_gui/class_configuration_file.py
View File

@ -60,8 +60,8 @@ class ConfigurationFile:
# Dropdown for selecting or entering the name of a configuration file.
self.config_file_name = gr.Dropdown(
label="Load/Save Config file",
choices=[""] + self.list_config_dir(self.current_config_dir),
value="",
choices=[self.config.get("config_dir", "")] + self.list_config_dir(self.current_config_dir),
value=self.config.get("config_dir", ""),
interactive=True,
allow_custom_value=True,
)

12
kohya_gui/class_folders.py
View File

@ -103,8 +103,8 @@ class Folders:
# Output directory dropdown
self.output_dir = gr.Dropdown(
label="Output directory for trained model",
choices=[""] + self.list_output_dirs(self.current_output_dir),
value="",
choices=[self.config.get("folders.output_dir", "")] + self.list_output_dirs(self.current_output_dir),
value=self.config.get("folders.output_dir", ""),
interactive=True,
allow_custom_value=True,
)
@ -138,8 +138,8 @@ class Folders:
if not self.finetune
else "Train config directory (Optional. where config files will be saved)"
),
choices=[""] + self.list_reg_data_dirs(self.current_reg_data_dir),
value="",
choices=[self.config.get("folders.reg_data_dir", "")] + self.list_reg_data_dirs(self.current_reg_data_dir),
value=self.config.get("folders.reg_data_dir", ""),
interactive=True,
allow_custom_value=True,
)
@ -169,8 +169,8 @@ class Folders:
# Logging directory dropdown
self.logging_dir = gr.Dropdown(
label="Logging directory (Optional. to enable logging and output Tensorboard log)",
choices=[""] + self.list_logging_dirs(self.current_logging_dir),
value="",
choices=[self.config.get("folders.logging_dir", "")] + self.list_logging_dirs(self.current_logging_dir),
value=self.config.get("folders.logging_dir", ""),
interactive=True,
allow_custom_value=True,
)

23
kohya_gui/class_source_model.py
View File

@ -57,13 +57,13 @@ class SourceModel:
# Set default directories if not provided
self.current_models_dir = self.config.get(
"models_dir", os.path.join(scriptdir, "models")
"model.models_dir", os.path.join(scriptdir, "models")
)
self.current_train_data_dir = self.config.get(
"train_data_dir", os.path.join(scriptdir, "data")
"model.train_data_dir", os.path.join(scriptdir, "data")
)
self.current_dataset_config_dir = self.config.get(
"dataset_config_dir", os.path.join(scriptdir, "dataset_config")
"model.dataset_config", os.path.join(scriptdir, "dataset_config")
)
model_checkpoints = list(
@ -82,7 +82,7 @@ class SourceModel:
def list_train_data_dirs(path):
self.current_train_data_dir = path if not path == "" else "."
return list(list_dirs(path))
return list(list_dirs(self.current_train_data_dir))
def list_dataset_config_dirs(path: str) -> list:
"""
@ -109,7 +109,7 @@ class SourceModel:
self.pretrained_model_name_or_path = gr.Dropdown(
label="Pretrained model name or path",
choices=default_models + model_checkpoints,
value="runwayml/stable-diffusion-v1-5",
value=self.config.get("model.models_dir", "runwayml/stable-diffusion-v1-5"),
allow_custom_value=True,
visible=True,
min_width=100,
@ -150,7 +150,7 @@ class SourceModel:
self.output_name = gr.Textbox(
label="Trained Model output name",
placeholder="(Name of the model to output)",
value="last",
value=self.config.get("model.output_name", "last"),
interactive=True,
)
with gr.Row():
@ -163,7 +163,7 @@ class SourceModel:
),
choices=[""]
+ list_train_data_dirs(self.current_train_data_dir),
value="",
value=self.config.get("model.train_data_dir", ""),
interactive=True,
allow_custom_value=True,
)
@ -191,9 +191,9 @@ class SourceModel:
# Toml directory dropdown
self.dataset_config = gr.Dropdown(
label="Dataset config file (Optional. Select the toml configuration file to use for the dataset)",
choices=[""]
choices=[self.config.get("model.dataset_config", "")]
+ list_dataset_config_dirs(self.current_dataset_config_dir),
value="",
value=self.config.get("model.dataset_config", ""),
interactive=True,
allow_custom_value=True,
)
@ -264,18 +264,19 @@ class SourceModel:
label="Training comment",
placeholder="(Optional) Add training comment to be included in metadata",
interactive=True,
value=self.config.get("model.training_comment", ""),
)
with gr.Row():
self.save_model_as = gr.Radio(
save_model_as_choices,
label="Save trained model as",
value="safetensors",
value=self.config.get("model.save_model_as", "safetensors"),
)
self.save_precision = gr.Radio(
save_precision_choices,
label="Save precision",
value="fp16",
value=self.config.get("model.save_precision", "fp16"),
)
self.pretrained_model_name_or_path.change(