v25.0.3

- Upgrade Gradio, diffusers and huggingface-hub to latest release to fix issue with ASGI. - Add a new method to setup and run the GUI. You will find two new script for both Windows (gui-uv.bat) and Linux (gui-uv.sh). With those scripts there is no need to run setup.bat or setup.sh anymore.
2025-03-28 17:46:38 -04:00 · 2025-03-28 17:46:38 -04:00 · b9028e8710
parent b65b52cf30
commit b9028e8710
15 changed files with 2816 additions and 62 deletions
--- a/.gitignore
+++ b/.gitignore
@ -1,4 +1,5 @@
 # Python
+.venv
 venv
 venv2
 __pycache__
--- a/.python-version
+++ b/.python-version
@ -0,0 +1 @@
+3.10
--- a/.release
+++ b/.release
@ -1 +1 @@
-v25.0.2
+v25.0.3
--- a/README.md
+++ b/README.md
@ -35,8 +35,10 @@ The GUI allows you to set the training parameters and generate and run the requi
    - [Windows Upgrade](#windows-upgrade)
    - [Linux and macOS Upgrade](#linux-and-macos-upgrade)
  - [Starting GUI Service](#starting-gui-service)
-    - [Launching the GUI on Windows](#launching-the-gui-on-windows)
+    - [Launching the GUI on Windows (non uv based method)](#launching-the-gui-on-windows-non-uv-based-method)
+    - [Alternative: UV-based Method](#alternative-uv-based-method)
    - [Launching the GUI on Linux and macOS](#launching-the-gui-on-linux-and-macos)
+    - [Alternative: UV-based Method for Linux](#alternative-uv-based-method-for-linux)
  - [Custom Path Defaults](#custom-path-defaults)
  - [LoRA](#lora)
  - [Sample image generation during training](#sample-image-generation-during-training)
@ -55,6 +57,7 @@ The GUI allows you to set the training parameters and generate and run the requi
      - [Monitoring Processes](#monitoring-processes)
  - [Interesting Forks](#interesting-forks)
  - [Change History](#change-history)
+    - [v25.0.3](#v2503)
    - [v25.0.2](#v2502)
    - [v25.0.1](#v2501)
    - [v25.0.0](#v2500)
@ -104,7 +107,9 @@ To set up the project, follow these steps:
   cd kohya_ss
   ```

-4. Run one of the following setup script by executing the following command:
+4. If you want to use the new uv based version of the script to run the GUI, you do not need to follow this step. On the other hand, if you want to use the legacy "pip" based method, please follow this next step.
+
+   Run one of the following setup script by executing the following command:

   For systems with only python 3.10.11 installed:

@ -160,13 +165,15 @@ To set up the project on Linux or macOS, perform the following steps:
   cd kohya_ss
   ```

-4. If you encounter permission issues, make the `setup.sh` script executable by running the following command:
+4. If you want to use the new uv based version of the script to run the GUI, you do not need to follow this step. On the other hand, if you want to use the legacy "pip" based method, please follow this next step.
+
+   If you encounter permission issues, make the `setup.sh` script executable by running the following command:

   ```shell
   chmod +x ./setup.sh
   ```

-5. Run the setup script by executing the following command:
+   Run the setup script by executing the following command:

   ```shell
   ./setup.sh
@ -321,7 +328,7 @@ If a new release becomes available, you can upgrade your repository by running t
   git pull
   ```

-2. Run the setup script:
+2. Run the setup script (if you do not use the uv script. If you use the uv versions of the scripts the updates are done for you automatically.):

   ```powershell
   .\setup.bat
@ -339,7 +346,7 @@ To upgrade your installation on Linux or macOS, follow these steps:
   git pull
   ```

-3. Refresh and update everything:
+3. Refresh and update everything (if you do not use the uv based script. If you use the uv versions of the scripts the updates are done for you automatically.):

   ```bash
   ./setup.sh
@ -373,7 +380,7 @@ To launch the GUI service, you can use the provided scripts or run the `kohya_gu
  --noverify            Disable requirements verification
 ```

-### Launching the GUI on Windows
+### Launching the GUI on Windows (non uv based method)

 On Windows, you can use either the `gui.ps1` or `gui.bat` script located in the root directory. Choose the script that suits your preference and run it in a terminal, providing the desired command line arguments. Here's an example:

@ -387,14 +394,46 @@ or
 gui.bat --listen 127.0.0.1 --server_port 7860 --inbrowser --share
 ```

+### Alternative: UV-based Method
+
+To use the UV-based method for running the GUI, which does not require running `setup.bat`, follow these steps:
+
+```cmd
+.\gui-uv.bat
+```
+
+or
+
+```powershell
+.\gui-uv.bat --listen 127.0.0.1 --server_port 7860 --inbrowser --share
+```
+
+This method provides an alternative way to run the GUI with the latest features, including support for Flux.1 and SD3, and eliminates the need for the setup script.
+
 ### Launching the GUI on Linux and macOS

 To launch the GUI on Linux or macOS, run the `gui.sh` script located in the root directory. Provide the desired command line arguments as follows:

 ```bash
-gui.sh --listen 127.0.0.1 --server_port 7860 --inbrowser --share
+./gui.sh --listen 127.0.0.1 --server_port 7860 --inbrowser --share
 ```

+### Alternative: UV-based Method for Linux
+
+To use the UV-based method for running the GUI, which does not require running `setup.sh`, follow these steps:
+
+```shell
+./gui-uv.sh --listen 127.0.0.1 --server_port 7860 --inbrowser --share
+```
+
+If you are running on a headless server, use:
+
+```shell
+./gui-uv.sh --headless --listen 127.0.0.1 --server_port 7860 --inbrowser --share
+```
+
+This method provides an alternative way to run the GUI with the latest features, including support for Flux.1 and SD3, and eliminates the need for the setup script.
+
 ## Custom Path Defaults

 The repository now provides a default configuration file named `config.toml`. This file is a template that you can customize to suit your needs.
@ -504,6 +543,11 @@ To finetune HunyuanDiT models or create LoRAs, visit this [fork](https://github.

 ## Change History

+### v25.0.3
+
+- Upgrade Gradio, diffusers and huggingface-hub to latest release to fix issue with ASGI.
+- Add a new method to setup and run the GUI. You will find two new script for both Windows (gui-uv.bat) and Linux (gui-uv.sh). With those scripts there is no need to run setup.bat or setup.sh anymore.
+
 ### v25.0.2

 - Force gradio to 5.14.0 or greater so it is updated.
--- a/example.toml
+++ b/example.toml
@ -2,7 +2,7 @@
 # Edit the values to suit your needs

 [settings]
-use_shell = false # Use shell furing process run of sd-scripts oython code. Most secure is false but some systems may require it to be true to properly run sd-scripts.
+use_shell = false # Use shell during process run of sd-scripts oython code. Most secure is false but some systems may require it to be true to properly run sd-scripts.

 # Default folders location
 [model]
--- a/gui-uv.bat
+++ b/gui-uv.bat
@ -0,0 +1,25 @@
+@echo off
+set VIRTUAL_ENV=.venv
+echo VIRTUAL_ENV is set to %VIRTUAL_ENV%
+
+:: Install uv latest version
+pip install --upgrade uv -q
+
+set PATH=%PATH%;%~dp0venv\Lib\site-packages\torch\lib
+
+echo Starting the GUI... this might take some time... Especially on 1st run after install or update...
+
+:: Make sure we are on the right sd-scripts commit
+git submodule update --init --recursive
+
+:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
+if %errorlevel% equ 0 (
+    REM Check if the batch was started via double-click
+    IF /i "%comspec% /c %~0 " equ "%cmdcmdline:"=%" (
+        REM echo This script was started by double clicking.
+        cmd /k uv run --link-mode=copy --index-strategy unsafe-best-match kohya_gui.py --noverify %*
+    ) ELSE (
+        REM echo This script was started from a command prompt.
+        uv run --link-mode=copy --index-strategy unsafe-best-match kohya_gui.py --noverify %*
+    )
+)
--- a/gui-uv.sh
+++ b/gui-uv.sh
@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+export VIRTUAL_ENV=.venv
+
+# Checks to see if variable is set and non-empty.
+# This is defined first, so we can use the function for some default variable values
+env_var_exists() {
+  if [[ -n "${!1}" ]]; then
+    return 0
+  else
+    return 1
+  fi
+}
+
+# Define the directory path for WSL2
+lib_path="/usr/lib/wsl/lib/"
+
+# Check if the directory exists
+if [ -d "$lib_path" ]; then
+    # Check if LD_LIBRARY_PATH is already set
+    if [ -z "${LD_LIBRARY_PATH}" ]; then
+        # LD_LIBRARY_PATH is not set, set it to the lib_path
+        export LD_LIBRARY_PATH="$lib_path"
+        # echo "LD_LIBRARY_PATH set to: $LD_LIBRARY_PATH"
+    fi
+fi
+
+# If it is run with the sudo command, get the complete LD_LIBRARY_PATH environment variable of the system and assign it to the current environment,
+# because it will be used later.
+if [ -n "$SUDO_USER" ] || [ -n "$SUDO_COMMAND" ]; then
+    echo "The sudo command resets the non-essential environment variables, we keep the LD_LIBRARY_PATH variable."
+    export LD_LIBRARY_PATH=$(sudo -i printenv LD_LIBRARY_PATH)
+fi
+
+# This gets the directory the script is run from so pathing can work relative to the script where needed.
+SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
+
+# Step into GUI local directory
+cd "$SCRIPT_DIR" || exit 1
+
+# Check if uv is already installed
+if ! command -v uv &> /dev/null; then
+    # Setup uv
+    curl -LsSf https://astral.sh/uv/install.sh | sh
+    source $HOME/.local/bin/env
+fi
+
+git submodule update --init --recursive
+uv run kohya_gui.py --noverify "$@"
--- a/gui.bat
+++ b/gui.bat
@ -15,6 +15,8 @@ set PATH=%PATH%;%~dp0venv\Lib\site-packages\torch\lib

 echo Starting the GUI... this might take some time...

+:skip_validation
+
 :: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
 if %errorlevel% equ 0 (
    REM Check if the batch was started via double-click
--- a/kohya_gui/class_flux1.py
+++ b/kohya_gui/class_flux1.py
@ -212,7 +212,7 @@ class flux1Training:
                    #     interactive=True,
                    # )
                    self.single_blocks_to_swap = gr.Slider(
-                        label="Single Blocks to swap (depercated)",
+                        label="Single Blocks to swap (deprecated)",
                        value=self.config.get("flux1.single_blocks_to_swap", 0),
                        info="[Experimental] Sets the number of 'single_blocks' (~320MB) to swap during the forward and backward passes.",
                        minimum=0,
@ -221,7 +221,7 @@ class flux1Training:
                        interactive=True,
                    )
                    self.double_blocks_to_swap = gr.Slider(
-                        label="Double Blocks to swap (depercated)",
+                        label="Double Blocks to swap (deprecated)",
                        value=self.config.get("flux1.double_blocks_to_swap", 0),
                        info="[Experimental] Sets the number of 'double_blocks' (~640MB) to swap during the forward and backward passes.",
                        minimum=0,
--- a/pyproject.toml
+++ b/pyproject.toml
@ -0,0 +1,58 @@
+[project]
+name = "kohya-ss"
+version = "0.1.0"
+description = "Add your description here"
+readme = "README.md"
+requires-python = ">=3.10,<3.11"
+dependencies = [
+    "accelerate==0.33.0",
+    "aiofiles==23.2.1",
+    "altair==4.2.2",
+    "bitsandbytes==0.44.0",
+    "dadaptation==3.2",
+    "diffusers[torch]==0.32.2",
+    "easygui==0.98.3",
+    "einops==0.7.0",
+    "fairscale==0.4.13",
+    "ftfy==6.1.1",
+    "gradio>=5.23.1",
+    "huggingface-hub==0.29.3",
+    "imagesize==1.4.1",
+    "invisible-watermark==0.2.0",
+    "library",
+    "lion-pytorch==0.0.6",
+    "lycoris-lora==3.1.0",
+    "omegaconf==2.3.0",
+    "onnx==1.16.1",
+    "onnxruntime-gpu==1.19.2",
+    "open-clip-torch==2.20.0",
+    "opencv-python==4.10.0.84",
+    "prodigy-plus-schedule-free==1.8.0",
+    "prodigyopt==1.1.2",
+    "protobuf==3.20.3",
+    "pytorch-lightning==1.9.0",
+    "pytorch-optimizer==3.5.0",
+    "rich>=13.7.1",
+    "safetensors==0.4.4",
+    "schedulefree==1.4",
+    "scipy==1.11.4",
+    "sentencepiece==0.2.0",
+    "tensorboard>=2.18.0",
+    "tensorflow>=2.16.1",
+    "tensorflow-io-gcs-filesystem==0.31.0",
+    "timm==0.6.12",
+    "tk==0.1.0",
+    "toml==0.10.2",
+    "torch>=2.5.0",
+    "torchvision>=0.20.0",
+    "transformers==4.44.2",
+    "voluptuous==0.13.1",
+    "wandb==0.18.0",
+    "xformers>=0.0.28.post2",
+]
+
+[tool.uv.sources]
+library = { path = "sd-scripts" }
+
+[[tool.uv.index]]
+url = "https://download.pytorch.org/whl/cu124"
--- a/requirements.txt
+++ b/requirements.txt
@ -2,13 +2,13 @@ accelerate==0.33.0
 aiofiles==23.2.1
 altair==4.2.2
 dadaptation==3.2
-diffusers[torch]==0.25.0
+diffusers[torch]==0.32.2
 easygui==0.98.3
 einops==0.7.0
 fairscale==0.4.13
 ftfy==6.1.1
-gradio>=5.14.0
-huggingface-hub==0.25.2
+gradio>=5.23.1
+huggingface-hub==0.29.3
 imagesize==1.4.1
 invisible-watermark==0.2.0
 lion-pytorch==0.0.6
--- a/setup-uv.bat
+++ b/setup-uv.bat
@ -0,0 +1,32 @@
+@echo off
+
+IF NOT EXIST venv (
+    echo Creating venv...
+    python -m venv venv
+)
+
+:: Create the directory if it doesn't exist
+mkdir ".\logs\setup" > nul 2>&1
+
+:: Deactivate the virtual environment to prevent error
+call .\venv\Scripts\deactivate.bat
+
+call .\venv\Scripts\activate.bat
+
+REM first make sure we have setuptools and uv available in the venv    
+python -m pip install --require-virtualenv --no-input -q -q  setuptools uv
+
+REM upgrade pip if needed
+python -m pip install --upgrade pip -q
+
+REM Check if the batch was started via double-click
+IF /i "%comspec% /c %~0 " equ "%cmdcmdline:"=%" (
+    REM echo This script was started by double clicking.
+    cmd /k python .\setup\setup_windows.py
+) ELSE (
+    REM echo This script was started from a command prompt.
+    python .\setup\setup_windows.py %*
+)
+
+:: Deactivate the virtual environment
+call .\venv\Scripts\deactivate.bat
--- a/setup/setup_common.py
+++ b/setup/setup_common.py
@ -162,6 +162,9 @@ def install_requirements_inbulk(

    # Build the command as a list
    cmd = ["pip", "install", "-r", requirements_file]
+    if installed("uv") and shutil.which("uv"):
+        log.info("Using uv for pip...")
+        cmd.insert(0, "uv")
    if upgrade:
        cmd.append("--upgrade")
    if not show_stdout:
@ -179,6 +182,9 @@ def install_requirements_inbulk(
        )

        for line in process.stdout:
+            if installed("uv"):
+                print(line)
+            else:
                if "Requirement already satisfied" not in line:
                    log.info(line.strip()) if show_stdout else None

@ -447,9 +453,13 @@ def pip(arg: str, ignore: bool = False, quiet: bool = False, show_stdout: bool =
        log.info(
            f'Installing package: {arg.replace("install", "").replace("--upgrade", "").replace("--no-deps", "").replace("--force", "").replace("  ", " ").strip()}'
        )
+    # pip_cmd = [rf"{sys.executable}", "-m", "pip"] + arg.split(" ")
    pip_cmd = [rf"{sys.executable}", "-m", "pip"] + arg.split(" ")
+    if installed("uv"):
+        log.info("Using uv for pip...")
+        pip_cmd.insert(2, "uv")
    log.debug(f"Running pip: {pip_cmd}")
-    if show_stdout:
+    if show_stdout or installed("uv"):
        subprocess.run(pip_cmd, shell=False, check=False, env=os.environ)
    else:
        result = subprocess.run(
@ -653,7 +663,7 @@ def ensure_base_requirements():
    try:
        import rich  # pylint: disable=unused-import
    except ImportError:
-        install("--upgrade rich", "rich")
+        install("rich", "rich")

    try:
        import packaging
--- a/test/config/SDXL-Standard-AdamW.json
+++ b/test/config/SDXL-Standard-AdamW.json
@ -1,106 +1,174 @@
 {
-  "LoRA_type": "Standard",
  "adaptive_noise_scale": 0,
  "additional_parameters": "",
-  "block_alphas": "",
-  "block_dims": "",
-  "block_lr_zero_threshold": "",
+  "ae": "",
+  "apply_t5_attn_mask": false,
+  "async_upload": false,
+  "blocks_to_swap": 0,
+  "blockwise_fused_optimizers": false,
  "bucket_no_upscale": false,
-  "bucket_reso_steps": 1,
+  "bucket_reso_steps": 32,
  "cache_latents": true,
  "cache_latents_to_disk": true,
-  "caption_dropout_every_n_epochs": 0.0,
+  "caption_dropout_every_n_epochs": 0,
  "caption_dropout_rate": 0,
-  "caption_extension": ".none-use-foldername",
-  "clip_skip": "1",
+  "caption_extension": ".txt",
+  "clip_g": "",
+  "clip_l": "",
+  "clip_skip": 1,
  "color_aug": false,
-  "conv_alpha": 64,
-  "conv_alphas": "",
-  "conv_dim": 64,
-  "conv_dims": "",
-  "decompose_both": false,
-  "dim_from_weights": false,
-  "down_lr_weight": "",
+  "cpu_offload_checkpointing": false,
+  "dataset_config": "",
+  "debiased_estimation_loss": false,
+  "disable_mmap_load_safetensors": false,
+  "discrete_flow_shift": 3,
+  "double_blocks_to_swap": 0,
+  "dynamo_backend": "no",
+  "dynamo_mode": "default",
+  "dynamo_use_dynamic": false,
+  "dynamo_use_fullgraph": false,
  "enable_bucket": true,
  "epoch": 1,
-  "factor": -1,
+  "extra_accelerate_launch_args": "",
  "flip_aug": false,
+  "flux1_cache_text_encoder_outputs": false,
+  "flux1_cache_text_encoder_outputs_to_disk": false,
+  "flux1_checkbox": false,
+  "flux1_clip_l": "",
+  "flux1_t5xxl": "",
+  "flux_fused_backward_pass": false,
+  "fp8_base": false,
+  "full_bf16": false,
  "full_fp16": false,
-  "gradient_accumulation_steps": 1.0,
+  "fused_backward_pass": false,
+  "fused_optimizer_groups": 0,
+  "gpu_ids": "",
+  "gradient_accumulation_steps": 1,
  "gradient_checkpointing": true,
-  "keep_tokens": "0",
+  "guidance_scale": 3.5,
+  "huber_c": 0.1,
+  "huber_scale": 1,
+  "huber_schedule": "snr",
+  "huggingface_path_in_repo": "",
+  "huggingface_repo_id": "",
+  "huggingface_repo_type": "",
+  "huggingface_repo_visibility": "",
+  "huggingface_token": "",
+  "ip_noise_gamma": 0,
+  "ip_noise_gamma_random_strength": false,
+  "keep_tokens": 0,
  "learning_rate": 0.0001,
+  "learning_rate_te": 1e-05,
+  "learning_rate_te1": 1e-05,
+  "learning_rate_te2": 1e-05,
+  "log_config": false,
+  "log_tracker_config": "",
+  "log_tracker_name": "",
+  "log_with": "",
  "logging_dir": "./test/logs",
-  "lora_network_weights": "",
+  "logit_mean": 0,
+  "logit_std": 1,
+  "loss_type": "l2",
  "lr_scheduler": "cosine",
-  "lr_scheduler_num_cycles": "1",
-  "lr_scheduler_power": "",
+  "lr_scheduler_args": "",
+  "lr_scheduler_num_cycles": 1,
+  "lr_scheduler_power": 1,
+  "lr_scheduler_type": "",
  "lr_warmup": 0,
-  "max_data_loader_n_workers": "0",
+  "lr_warmup_steps": 0,
+  "main_process_port": 0,
+  "masked_loss": false,
+  "max_bucket_reso": 2048,
+  "max_data_loader_n_workers": 0,
  "max_resolution": "1024,1024",
  "max_timestep": 1000,
-  "max_token_length": "75",
-  "max_train_epochs": "",
+  "max_token_length": 75,
+  "max_train_epochs": 0,
+  "max_train_steps": 1600,
  "mem_eff_attn": false,
-  "mid_lr_weight": "",
+  "mem_eff_save": false,
+  "metadata_author": "",
+  "metadata_description": "",
+  "metadata_license": "",
+  "metadata_tags": "",
+  "metadata_title": "",
+  "min_bucket_reso": 256,
  "min_snr_gamma": 10,
  "min_timestep": 0,
  "mixed_precision": "fp16",
+  "mode_scale": 1.29,
  "model_list": "custom",
-  "module_dropout": 0,
+  "model_prediction_type": "sigma_scaled",
+  "multi_gpu": false,
  "multires_noise_discount": 0.2,
  "multires_noise_iterations": 8,
-  "network_alpha": 64,
-  "network_dim": 64,
-  "network_dropout": 0,
  "no_token_padding": false,
  "noise_offset": 0.0357,
+  "noise_offset_random_strength": false,
  "noise_offset_type": "Original",
  "num_cpu_threads_per_process": 2,
+  "num_machines": 1,
+  "num_processes": 1,
  "optimizer": "AdamW",
  "optimizer_args": "",
  "output_dir": "./test/output",
  "output_name": "SDXL-Standard-AdamW",
  "persistent_data_loader_workers": false,
  "pretrained_model_name_or_path": "D:/models/sdxl/sd_xl_base_0.9-pruned.safetensors",
-  "prior_loss_weight": 1.0,
+  "prior_loss_weight": 1,
  "random_crop": false,
-  "rank_dropout": 0,
  "reg_data_dir": "",
  "resume": "",
+  "resume_from_huggingface": "",
  "sample_every_n_epochs": 0,
  "sample_every_n_steps": 0,
  "sample_prompts": "",
  "sample_sampler": "euler_a",
+  "save_as_bool": false,
+  "save_clip": false,
  "save_every_n_epochs": 1,
  "save_every_n_steps": 0,
+  "save_last_n_epochs": 0,
+  "save_last_n_epochs_state": 0,
  "save_last_n_steps": 0,
  "save_last_n_steps_state": 0,
  "save_model_as": "safetensors",
  "save_precision": "fp16",
  "save_state": false,
+  "save_state_on_train_end": false,
+  "save_state_to_huggingface": false,
+  "save_t5xxl": false,
  "scale_v_pred_loss_like_noise_pred": false,
-  "scale_weight_norms": 0,
+  "sd3_cache_text_encoder_outputs": false,
+  "sd3_cache_text_encoder_outputs_to_disk": false,
+  "sd3_checkbox": false,
+  "sd3_fused_backward_pass": false,
+  "sd3_text_encoder_batch_size": 1,
  "sdxl": true,
-  "sdxl_cache_text_encoder_outputs": true,
+  "sdxl_cache_text_encoder_outputs": false,
  "sdxl_no_half_vae": true,
-  "seed": "",
+  "seed": 0,
  "shuffle_caption": false,
+  "single_blocks_to_swap": 0,
+  "skip_cache_check": false,
+  "split_mode": false,
  "stop_text_encoder_training": 0,
-  "text_encoder_lr": 0.0,
+  "t5xxl": "",
+  "t5xxl_device": "",
+  "t5xxl_dtype": "bf16",
+  "t5xxl_max_token_length": 512,
+  "timestep_sampling": "sigma",
  "train_batch_size": 1,
+  "train_blocks": "all",
  "train_data_dir": "./test/img",
-  "train_on_input": true,
-  "training_comment": "",
-  "unet_lr": 0.0001,
-  "unit": 1,
-  "up_lr_weight": "",
-  "use_cp": false,
-  "use_wandb": false,
  "v2": false,
  "v_parameterization": false,
+  "v_pred_like_loss": 0,
+  "vae": "",
  "vae_batch_size": 0,
  "wandb_api_key": "",
+  "wandb_run_name": "",
  "weighted_captions": false,
-  "xformers": true
+  "weighting_scheme": "logit_normal",
+  "xformers": "xformers"
 }
--- a/uv.lock
+++ b/uv.lock