Remove torch 1 support from code

pull/1589/head^2
bmaltais 2023-10-04 19:00:12 -04:00
parent 1deb505922
commit 93a181f55b
6 changed files with 231 additions and 251 deletions

View File

@ -625,6 +625,9 @@ ControlNet-LLLite, a novel method for ControlNet with SDXL, is added. See [docum
## Change History ## Change History
* 2023/10/08 (v22.1.0)
- Remove support for torch 1 to align with kohya_ss sd-scripts code base.
* 2023/10/01 (v22.0.0) * 2023/10/01 (v22.0.0)
- Merging main branch of sd-scripts: - Merging main branch of sd-scripts:
- [SAI Model Spec](https://github.com/Stability-AI/ModelSpec) metadata is now supported partially. `hash_sha256` is not supported yet. - [SAI Model Spec](https://github.com/Stability-AI/ModelSpec) metadata is now supported partially. `hash_sha256` is not supported yet.
@ -648,16 +651,3 @@ ControlNet-LLLite, a novel method for ControlNet with SDXL, is added. See [docum
- Update wandb module version - Update wandb module version
- Add support for Chinese zh-CN localisation. You can use it with `.\gui.bat --language=zh-CN` - Add support for Chinese zh-CN localisation. You can use it with `.\gui.bat --language=zh-CN`
- Add presets support to `Finetuning`. You can add your own finetuning user presets under the `/presets/finetune/user_presets` folder. - Add presets support to `Finetuning`. You can add your own finetuning user presets under the `/presets/finetune/user_presets` folder.
* 2023/09/23 (v21.8.10)
- Minor point upgrade. Mostly adding a new preset.
* 2023/08/05 (v21.8.9)
- Update sd-script to caode as of Sept 3 2023
* ControlNet-LLLite is added. See documentation for details.
* JPEG XL is supported. #786
* Peak memory usage is reduced. #791
* Input perturbation noise is added. See #798 for details.
* Dataset subset now has caption_prefix and caption_suffix options. The strings are added to the beginning and the end of the captions before shuffling. You can specify the options in .toml.
* Other minor changes.
- Added support for Chinese locallisation

View File

@ -1,188 +1,188 @@
嗨!我把日语 README 文件的主要内容翻译成中文如下: 嗨!我把日语 README 文件的主要内容翻译成中文如下:
## 关于这个仓库 ## 关于这个仓库
这个是用于Stable Diffusion模型训练、图像生成和其他脚本的仓库。 这个是用于Stable Diffusion模型训练、图像生成和其他脚本的仓库。
[英文版 README](./README.md) <-- 更新信息在这里 [英文版 README](./README.md) <-- 更新信息在这里
GUI和PowerShell脚本等使其更易用的功能在[bmaltais的仓库](https://github.com/bmaltais/kohya_ss)(英语)中提供,一并参考。感谢bmaltais。 GUI和PowerShell脚本等使其更易用的功能在[bmaltais的仓库](https://github.com/bmaltais/kohya_ss)(英语)中提供,一并参考。感谢bmaltais。
包含以下脚本: 包含以下脚本:
* 支持DreamBooth、U-Net和文本编码器的训练 * 支持DreamBooth、U-Net和文本编码器的训练
* fine-tuning的支持 * fine-tuning的支持
* 图像生成 * 图像生成
* 模型转换(Stable Diffusion ckpt/safetensors 和 Diffusers之间的相互转换) * 模型转换(Stable Diffusion ckpt/safetensors 和 Diffusers之间的相互转换)
## 使用方法 (中国用户只需要按照这个安装教程操作) ## 使用方法 (中国用户只需要按照这个安装教程操作)
- 进入kohya_ss文件夹根目录下点击 setup.bat 启动安装程序 *(需要科学上网) - 进入kohya_ss文件夹根目录下点击 setup.bat 启动安装程序 *(需要科学上网)
- 根据界面上给出的英文选项: - 根据界面上给出的英文选项:
Kohya_ss GUI setup menu: Kohya_ss GUI setup menu:
1. Install kohya_ss gui 1. Install kohya_ss gui
2. (Optional) Install cudann files (avoid unless you really need it) 2. (Optional) Install cudann files (avoid unless you really need it)
3. (Optional) Install specific bitsandbytes versions 3. (Optional) Install specific bitsandbytes versions
4. (Optional) Manually configure accelerate 4. (Optional) Manually configure accelerate
5. (Optional) Start Kohya_ss GUI in browser 5. (Optional) Start Kohya_ss GUI in browser
6. Quit 6. Quit
Enter your choice: 1 Enter your choice: 1
1. Torch 1 (legacy, no longer supported. Will be removed in v21.9.x) 1. Torch 1 (legacy, no longer supported. Will be removed in v21.9.x)
2. Torch 2 (recommended) 2. Torch 2 (recommended)
3. Cancel 3. Cancel
Enter your choice: 2 Enter your choice: 2
开始安装环境依赖,接着再出来的选项,按照下列选项操作: 开始安装环境依赖,接着再出来的选项,按照下列选项操作:
```txt ```txt
- This machine - This machine
- No distributed training - No distributed training
- NO - NO
- NO - NO
- NO - NO
- all - all
- bf16 - bf16
``` ```
-------------------------------------------------------------------- --------------------------------------------------------------------
这里都选择完毕,即可关闭终端窗口,直接点击 gui.bat或者 kohya中文启动器.bat 即可运行kohya 这里都选择完毕,即可关闭终端窗口,直接点击 gui.bat或者 kohya中文启动器.bat 即可运行kohya
当仓库内和note.com有相关文章,请参考那里。(未来可能全部移到这里) 当仓库内和note.com有相关文章,请参考那里。(未来可能全部移到这里)
* [关于训练,通用篇](./docs/train_README-ja.md): 数据准备和选项等 * [关于训练,通用篇](./docs/train_README-ja.md): 数据准备和选项等
* [数据集设置](./docs/config_README-ja.md) * [数据集设置](./docs/config_README-ja.md)
* [DreamBooth训练指南](./docs/train_db_README-ja.md) * [DreamBooth训练指南](./docs/train_db_README-ja.md)
* [fine-tuning指南](./docs/fine_tune_README_ja.md) * [fine-tuning指南](./docs/fine_tune_README_ja.md)
* [LoRA训练指南](./docs/train_network_README-ja.md) * [LoRA训练指南](./docs/train_network_README-ja.md)
* [文本反转训练指南](./docs/train_ti_README-ja.md) * [文本反转训练指南](./docs/train_ti_README-ja.md)
* [图像生成脚本](./docs/gen_img_README-ja.md) * [图像生成脚本](./docs/gen_img_README-ja.md)
* note.com [模型转换脚本](https://note.com/kohya_ss/n/n374f316fe4ad) * note.com [模型转换脚本](https://note.com/kohya_ss/n/n374f316fe4ad)
## Windows环境所需程序 ## Windows环境所需程序
需要Python 3.10.6和Git。 需要Python 3.10.6和Git。
- Python 3.10.6: https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe - Python 3.10.6: https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe
- git: https://git-scm.com/download/win - git: https://git-scm.com/download/win
如果要在PowerShell中使用venv,需要按以下步骤更改安全设置: 如果要在PowerShell中使用venv,需要按以下步骤更改安全设置:
(不仅仅是venv,使脚本可以执行。请注意。) (不仅仅是venv,使脚本可以执行。请注意。)
- 以管理员身份打开PowerShell - 以管理员身份打开PowerShell
- 输入"Set-ExecutionPolicy Unrestricted",选择Y - 输入"Set-ExecutionPolicy Unrestricted",选择Y
- 关闭管理员PowerShell - 关闭管理员PowerShell
## 在Windows环境下安装 ## 在Windows环境下安装
下例中安装的是PyTorch 1.12.1/CUDA 11.6版。如果要使用CUDA 11.3或PyTorch 1.13,请适当修改。 下例中安装的是PyTorch 1.12.1/CUDA 11.6版。如果要使用CUDA 11.3或PyTorch 1.13,请适当修改。
(如果只显示"python",请将下例中的"python"改为"py") (如果只显示"python",请将下例中的"python"改为"py")
在普通(非管理员)PowerShell中依次执行以下命令: 在普通(非管理员)PowerShell中依次执行以下命令:
```powershell ```powershell
git clone https://github.com/kohya-ss/sd-scripts.git git clone https://github.com/kohya-ss/sd-scripts.git
cd sd-scripts cd sd-scripts
python -m venv venv python -m venv venv
.\venv\Scripts\activate .\venv\Scripts\activate
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116 pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116
pip install --upgrade -r requirements.txt pip install --upgrade -r requirements.txt
pip install -U -I --no-deps https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl pip install -U -I --no-deps https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl
cp .\bitsandbytes_windows\*.dll .\venv\Lib\site-packages\bitsandbytes\ cp .\bitsandbytes_windows\*.dll .\venv\Lib\site-packages\bitsandbytes\
cp .\bitsandbytes_windows\cextension.py .\venv\Lib\site-packages\bitsandbytes\cextension.py cp .\bitsandbytes_windows\cextension.py .\venv\Lib\site-packages\bitsandbytes\cextension.py
cp .\bitsandbytes_windows\main.py .\venv\Lib\site-packages\bitsandbytes\cuda_setup\main.py cp .\bitsandbytes_windows\main.py .\venv\Lib\site-packages\bitsandbytes\cuda_setup\main.py
accelerate config accelerate config
``` ```
在命令提示符中: 在命令提示符中:
```bat ```bat
git clone https://github.com/kohya-ss/sd-scripts.git git clone https://github.com/kohya-ss/sd-scripts.git
cd sd-scripts cd sd-scripts
python -m venv venv python -m venv venv
.\venv\Scripts\activate .\venv\Scripts\activate
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116 pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116
pip install --upgrade -r requirements.txt pip install --upgrade -r requirements.txt
pip install -U -I --no-deps https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl pip install -U -I --no-deps https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl
copy /y .\bitsandbytes_windows\*.dll .\venv\Lib\site-packages\bitsandbytes\ copy /y .\bitsandbytes_windows\*.dll .\venv\Lib\site-packages\bitsandbytes\
copy /y .\bitsandbytes_windows\cextension.py .\venv\Lib\site-packages\bitsandbytes\cextension.py copy /y .\bitsandbytes_windows\cextension.py .\venv\Lib\site-packages\bitsandbytes\cextension.py
copy /y .\bitsandbytes_windows\main.py .\venv\Lib\site-packages\bitsandbytes\cuda_setup\main.py copy /y .\bitsandbytes_windows\main.py .\venv\Lib\site-packages\bitsandbytes\cuda_setup\main.py
accelerate config accelerate config
``` ```
accelerate config的问题请按以下回答: accelerate config的问题请按以下回答:
(如果要用bf16训练,最后一个问题选择bf16) (如果要用bf16训练,最后一个问题选择bf16)
``` ```
- 此计算机 - 此计算机
- 不进行分布式训练 - 不进行分布式训练
- 否 - 否
- 否 - 否
- 否 - 否
- 所有 - 所有
- fp16 - fp16
``` ```
### PyTorch和xformers版本注意事项 ### PyTorch和xformers版本注意事项
在其他版本中训练可能失败。如果没有特殊原因,请使用指定版本。 在其他版本中训练可能失败。如果没有特殊原因,请使用指定版本。
### 可选:使用Lion8bit ### 可选:使用Lion8bit
如果要使用Lion8bit,需要将`bitsandbytes`升级到0.38.0以上。首先卸载`bitsandbytes`,然后在Windows中安装适合Windows的whl文件,例如[这里的](https://github.com/jllllll/bitsandbytes-windows-webui)。例如: 如果要使用Lion8bit,需要将`bitsandbytes`升级到0.38.0以上。首先卸载`bitsandbytes`,然后在Windows中安装适合Windows的whl文件,例如[这里的](https://github.com/jllllll/bitsandbytes-windows-webui)。例如:
```powershell ```powershell
pip install https://github.com/jllllll/bitsandbytes-windows-webui/raw/main/bitsandbytes-0.38.1-py3-none-any.whl pip install https://github.com/jllllll/bitsandbytes-windows-webui/raw/main/bitsandbytes-0.38.1-py3-none-any.whl
``` ```
升级时用`pip install .`更新这个仓库,并视情况升级其他包。 升级时用`pip install .`更新这个仓库,并视情况升级其他包。
### 可选:使用PagedAdamW8bit和PagedLion8bit ### 可选:使用PagedAdamW8bit和PagedLion8bit
如果要使用PagedAdamW8bit和PagedLion8bit,需要将`bitsandbytes`升级到0.39.0以上。首先卸载`bitsandbytes`,然后在Windows中安装适合Windows的whl文件,例如[这里的](https://github.com/jllllll/bitsandbytes-windows-webui)。例如: 如果要使用PagedAdamW8bit和PagedLion8bit,需要将`bitsandbytes`升级到0.39.0以上。首先卸载`bitsandbytes`,然后在Windows中安装适合Windows的whl文件,例如[这里的](https://github.com/jllllll/bitsandbytes-windows-webui)。例如:
```powershell ```powershell
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl
``` ```
升级时用`pip install .`更新这个仓库,并视情况升级其他包。 升级时用`pip install .`更新这个仓库,并视情况升级其他包。
## 升级 ## 升级
如果有新版本,可以用以下命令更新: 如果有新版本,可以用以下命令更新:
```powershell ```powershell
cd sd-scripts cd sd-scripts
git pull git pull
.\venv\Scripts\activate .\venv\Scripts\activate
pip install --use-pep517 --upgrade -r requirements.txt pip install --use-pep517 --upgrade -r requirements.txt
``` ```
如果命令成功,就可以使用新版本了。 如果命令成功,就可以使用新版本了。
## 致谢 ## 致谢
LoRA实现基于[cloneofsimo的仓库](https://github.com/cloneofsimo/lora)。表示感谢。 LoRA实现基于[cloneofsimo的仓库](https://github.com/cloneofsimo/lora)。表示感谢。
将Conv2d 3x3扩展到所有层起初由 [cloneofsimo](https://github.com/cloneofsimo/lora) 发布, [KohakuBlueleaf](https://github.com/KohakuBlueleaf/LoCon) 证明了其有效性。深深感谢 KohakuBlueleaf。 将Conv2d 3x3扩展到所有层起初由 [cloneofsimo](https://github.com/cloneofsimo/lora) 发布, [KohakuBlueleaf](https://github.com/KohakuBlueleaf/LoCon) 证明了其有效性。深深感谢 KohakuBlueleaf。
## 许可 ## 许可
脚本遵循 ASL 2.0 许可,但包含其他许可的代码部分(Diffusers和cloneofsimo的仓库)。 脚本遵循 ASL 2.0 许可,但包含其他许可的代码部分(Diffusers和cloneofsimo的仓库)。
[Memory Efficient Attention Pytorch](https://github.com/lucidrains/memory-efficient-attention-pytorch): MIT [Memory Efficient Attention Pytorch](https://github.com/lucidrains/memory-efficient-attention-pytorch): MIT
[bitsandbytes](https://github.com/TimDettmers/bitsandbytes): MIT [bitsandbytes](https://github.com/TimDettmers/bitsandbytes): MIT
[BLIP](https://github.com/salesforce/BLIP): BSD-3-Clause [BLIP](https://github.com/salesforce/BLIP): BSD-3-Clause

View File

@ -1,8 +0,0 @@
@echo off
call gui.bat --language zh-CN --inbrowser --share --headless
echo 完成。
pause
endlocal

View File

@ -13,7 +13,7 @@ huggingface-hub==0.15.1
# for loading Diffusers' SDXL # for loading Diffusers' SDXL
invisible-watermark==0.2.0 invisible-watermark==0.2.0
lion-pytorch==0.0.6 lion-pytorch==0.0.6
lycoris_lora==1.8.3 lycoris_lora==1.9.0
# for BLIP captioning # for BLIP captioning
# requests==2.28.2 # requests==2.28.2
# timm==0.6.12 # timm==0.6.12

View File

@ -98,31 +98,31 @@ def sync_bits_and_bytes_files():
log.error(f'An unexpected error occurred: {e}') log.error(f'An unexpected error occurred: {e}')
def install_kohya_ss_torch1(): # def install_kohya_ss_torch1():
setup_common.check_repo_version() # setup_common.check_repo_version()
setup_common.check_python() # setup_common.check_python()
# Upgrade pip if needed # # Upgrade pip if needed
setup_common.install('--upgrade pip') # setup_common.install('--upgrade pip')
if setup_common.check_torch() == 2: # if setup_common.check_torch() == 2:
input( # input(
f'{YELLOW}\nTorch 2 is already installed in the venv. To install Torch 1 delete the venv and re-run setup.bat\n\nHit enter to continue...{RESET_COLOR}' # f'{YELLOW}\nTorch 2 is already installed in the venv. To install Torch 1 delete the venv and re-run setup.bat\n\nHit enter to continue...{RESET_COLOR}'
) # )
return # return
# setup_common.install( # # setup_common.install(
# 'torch==1.12.1+cu116 torchvision==0.13.1+cu116 --index-url https://download.pytorch.org/whl/cu116', # # 'torch==1.12.1+cu116 torchvision==0.13.1+cu116 --index-url https://download.pytorch.org/whl/cu116',
# 'torch torchvision' # # 'torch torchvision'
# ) # # )
# setup_common.install( # # setup_common.install(
# 'https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl -U -I --no-deps', # # 'https://github.com/C43H66N12O12S2/stable-diffusion-webui/releases/download/f/xformers-0.0.14.dev0-cp310-cp310-win_amd64.whl -U -I --no-deps',
# 'xformers-0.0.14' # # 'xformers-0.0.14'
# ) # # )
setup_common.install_requirements('requirements_windows_torch1.txt', check_no_verify_flag=False) # setup_common.install_requirements('requirements_windows_torch1.txt', check_no_verify_flag=False)
sync_bits_and_bytes_files() # sync_bits_and_bytes_files()
setup_common.configure_accelerate(run_accelerate=True) # setup_common.configure_accelerate(run_accelerate=True)
# run_cmd(f'accelerate config') # # run_cmd(f'accelerate config')
def install_kohya_ss_torch2(): def install_kohya_ss_torch2():
@ -177,23 +177,24 @@ def main_menu():
print('') print('')
if choice == '1': if choice == '1':
while True: install_kohya_ss_torch2()
print('1. Torch 1 (legacy, no longer supported. Will be removed in v21.9.x)') # while True:
print('2. Torch 2 (recommended)') # print('1. Torch 1 (legacy, no longer supported. Will be removed in v21.9.x)')
print('3. Cancel') # print('2. Torch 2 (recommended)')
choice_torch = input('\nEnter your choice: ') # print('3. Cancel')
print('') # choice_torch = input('\nEnter your choice: ')
# print('')
if choice_torch == '1': # if choice_torch == '1':
install_kohya_ss_torch1() # install_kohya_ss_torch1()
break # break
elif choice_torch == '2': # elif choice_torch == '2':
install_kohya_ss_torch2() # install_kohya_ss_torch2()
break # break
elif choice_torch == '3': # elif choice_torch == '3':
break # break
else: # else:
print('Invalid choice. Please enter a number between 1-3.') # print('Invalid choice. Please enter a number between 1-3.')
elif choice == '2': elif choice == '2':
cudann_install() cudann_install()
elif choice == '3': elif choice == '3':

View File

@ -110,10 +110,7 @@ def main():
if args.requirements: if args.requirements:
setup_common.install_requirements(args.requirements, check_no_verify_flag=True) setup_common.install_requirements(args.requirements, check_no_verify_flag=True)
else: else:
if torch_ver == 1: setup_common.install_requirements('requirements_windows_torch2.txt', check_no_verify_flag=True)
setup_common.install_requirements('requirements_windows_torch1.txt', check_no_verify_flag=True)
else:
setup_common.install_requirements('requirements_windows_torch2.txt', check_no_verify_flag=True)
if __name__ == '__main__': if __name__ == '__main__':