mirror of https://github.com/vladmandic/automatic
new installer
Vladimir Mandic
parent
44b12c66a5
commit
831f562394
|
|
@ -2,6 +2,7 @@
|
|||
__pycache__
|
||||
/params.txt
|
||||
/cache.json
|
||||
/setup.log
|
||||
|
||||
# all models and temp files
|
||||
*.ckpt
|
||||
|
|
|
|||
107
README.md
107
README.md
|
|
@ -44,7 +44,6 @@ All code changes are merged upstream whenever possible
|
|||
|
||||
- Drops compatibility with `python` **3.7** and requires **3.9**
|
||||
Recommended is **Python 3.10**
|
||||
Note that **Python 3.11** or **3.12** are NOT supported
|
||||
- Drops localizations
|
||||
- Drops automated tests
|
||||
|
||||
|
|
@ -74,110 +73,42 @@ Fork adds extra functionality:
|
|||
|
||||
<br>
|
||||
|
||||
### Start Script
|
||||
|
||||
Simplified start script: `automatic.sh`
|
||||
*Existing `webui.sh`/`webui.bat` scripts still exist for backward compatibility*
|
||||
|
||||
> ./automatic.sh
|
||||
|
||||
Start in default mode with optimizations enabled
|
||||
|
||||
Stable Diffusion server: optimized
|
||||
Version: a4d00060 Sun Mar 26 10:28:05 2023 -0400
|
||||
Repository: https://github.com/vladmandic/automatic
|
||||
Platform: Ubuntu 22.04.2 LTS 5.15.90.1-microsoft-standard-WSL2 x86_64
|
||||
Installing requirements for Web UI
|
||||
Launching Web UI with arguments: --cors-allow-origins=http://127.0.0.1:7860 --ckpt models/v1-5-pruned-emaonly.safetensors
|
||||
Torch 2.0.0+cu118 CUDA 11.8 cuDNN 8700
|
||||
GPU NVIDIA GeForce RTX 3060 VRAM 12288 Arch (8, 6) Cores 28
|
||||
Running on local URL: http://127.0.0.1:7860
|
||||
Loading weights: models/v1-5-pruned-emaonly.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/4.3 GB -:--:--
|
||||
Creating model from config: /home/vlado/dev/automatic/configs/v1-inference.yaml
|
||||
DiffusionWrapper has 859.52 M params.
|
||||
Loading weights: models/VAE/vae-ft-mse-840000-ema-pruned.ckpt ━━━━━━━━━━━━━━━━━━━━━━━ 0.0/334.7 MB -:--:--
|
||||
Applying scaled dot product cross attention optimization.
|
||||
Textual inversion embeddings loaded(2): ti-mia, ti-vlado
|
||||
Model loaded in 1.6s (load weights: 0.1s, create model: 0.3s, apply weights: 0.4s, load vae: 0.3s, device move: 0.5s).
|
||||
Startup time: 11.8s (import torch: 1.7s, import libraries: 1.0s, list models: 1.9s, load scripts: 1.0s, create ui: 4.4s, load checkpoint: 1.7s).
|
||||
Progress 6.55it/s ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:04
|
||||
|
||||
> ./automatic.sh clean
|
||||
|
||||
Start with all optimizations disabled
|
||||
Use this for troubleshooting
|
||||
|
||||
> ./automatic.sh install
|
||||
|
||||
Installs and updates to latest supported version:
|
||||
|
||||
- Dependencies
|
||||
- Fixed sub-repositories
|
||||
- Extensions
|
||||
- Sub-modules
|
||||
|
||||
Does not update main repository
|
||||
|
||||
> ./automatic.sh update
|
||||
|
||||
Updates the main repository to the latest version
|
||||
Recommended to run `install` after `update` to update dependencies as they may have changed
|
||||
|
||||
> ./automatic.sh help
|
||||
|
||||
Print all available options
|
||||
|
||||
> ./automatic.sh public
|
||||
|
||||
Start with listen on public IP with authentication enabled
|
||||
|
||||
<br>
|
||||
|
||||
## Install
|
||||
|
||||
|
||||
### Install
|
||||
|
||||
1. Install `Python`, `Git`
|
||||
2. Install `PyTorch`
|
||||
See [Wiki](wiki/Torch%20Optimizations.md) for details or TL;DR below
|
||||
3. Clone and initialize repository
|
||||
2. Clone repository
|
||||
|
||||
> git clone https://github.com/vladmandic/automatic
|
||||
> cd automatic
|
||||
> ./automatic.sh install
|
||||
|
||||
SD server: install
|
||||
Version: 56f779a9 Sat Feb 25 14:04:19 2023 -0500
|
||||
Repository: https://github.com/vladmandic/automatic
|
||||
Last Merge: Sun Feb 19 10:11:25 2023 -0500 Merge pull request #37 from AUTOMATIC1111/master
|
||||
Installing general requirements
|
||||
Installing versioned requirements
|
||||
Installing requirements for Web UI
|
||||
Updating submodules
|
||||
Updating extensions
|
||||
Updating wiki
|
||||
Detached repos
|
||||
Local changes
|
||||
### Run
|
||||
|
||||
*Note*: If you're not using `automatic.sh` launcher, install dependencies manually:
|
||||
Run desired startup script to install dependencies and extensions and start server:
|
||||
|
||||
- `launch.py`:
|
||||
Main startup script
|
||||
Run `python launch.py --help` for available options
|
||||
- `launch-venv.bat` and `launch.venv.sh`:
|
||||
Platform specific wrapper scripts that starts `launch.py` in Python virtual environment
|
||||
*Note*: Server can run without virtual environment, but it is recommended to use it
|
||||
**If you're unsure which launcher to use, this is the one you want**
|
||||
- `setup.py`:
|
||||
Main installer, used by `launch.py`
|
||||
Can also be used directly to update repository or extensions
|
||||
Run `python setup.py --help` for available options
|
||||
Setup details are logged to `setup.log`
|
||||
|
||||
> pip -r requirements.txt
|
||||
> pip -r requirements_versions.txt
|
||||
|
||||
<br>
|
||||
|
||||
## Other
|
||||
|
||||
### Torch
|
||||
|
||||
Only Python library which is not auto-updated is `PyTorch` itself as that is very system specific
|
||||
Fork is compatible with regular **PyTorch 1.13**, **PyTorch 2.0** as well as pre-releases of **PyTorch** **2.1**
|
||||
TL;DR: Install **PyTorch 2.0.0** compiled with **CUDA 11.8**:
|
||||
|
||||
> pip install torch torchaudio torchvision triton --force --extra-index-url https://download.pytorch.org/whl/cu118
|
||||
|
||||
See [Wiki](https://github.com/vladmandic/automatic/wiki/Torch-Optimizations) for **Torch** optimization notes
|
||||
|
||||
<br>
|
||||
|
||||
### Scripts
|
||||
|
||||
This repository comes with a large collection of scripts that can be used to process inputs, train, generate, and benchmark models
|
||||
|
|
|
|||
10
TODO.md
10
TODO.md
|
|
@ -5,6 +5,16 @@
|
|||
Stuff to be fixed...
|
||||
|
||||
- Reconnect WebUI
|
||||
- Skip Torch
|
||||
- Update requirements
|
||||
- Update README
|
||||
- Fix CSS
|
||||
- Fix Firefox
|
||||
- Implement installer in Python
|
||||
- piexif, rich, kornia
|
||||
- Move cross-optimization to settings
|
||||
|
||||
- Fix mediapipe
|
||||
|
||||
## Integration
|
||||
|
||||
|
|
|
|||
182
automatic.sh
182
automatic.sh
|
|
@ -1,182 +0,0 @@
|
|||
#!/bin/env bash
|
||||
|
||||
export TF_CPP_MIN_LOG_LEVEL=2
|
||||
export ACCELERATE="True"
|
||||
export FORCE_CUDA="1"
|
||||
export ATTN_PRECISION=fp16
|
||||
export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512
|
||||
export CUDA_LAUNCH_BLOCKING=0
|
||||
export CUDA_CACHE_DISABLE=0
|
||||
export CUDA_AUTO_BOOST=1
|
||||
export CUDA_MODULE_LOADING="LAZY"
|
||||
export CUDA_DEVICE_DEFAULT_PERSISTING_L2_CACHE_PERCENTAGE_LIMIT=0
|
||||
export GRADIO_ANALYTICS_ENABLED="False"
|
||||
export SAFETENSORS_FAST_GPU=1
|
||||
|
||||
# Check for Python
|
||||
if [ "$PYTHON" == "" ]; then
|
||||
PYTHON=$(which python)
|
||||
else
|
||||
echo "Python env variable set: $PYTHON"
|
||||
fi
|
||||
if [ "$PYTHON" == "" ]; then
|
||||
PYTHON=$(which python3)
|
||||
fi
|
||||
if ! type -P "$PYTHON" >/dev/null; then
|
||||
echo "Python not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Note: Some defaults are changed in shared.py
|
||||
CMD="launch.py --cors-allow-origins=http://127.0.0.1:7860"
|
||||
|
||||
MODE=optimized
|
||||
|
||||
# Sanity checks
|
||||
if [[ $(id -u) -eq 0 ]]; then
|
||||
echo "Running as root, aborting"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
"$PYTHON" -m pip --quiet show torch
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Torch not installed, aborting"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Parse arguments
|
||||
for i in "$@"; do
|
||||
case $i in
|
||||
update)
|
||||
MODE=update
|
||||
;;
|
||||
install)
|
||||
MODE=install
|
||||
;;
|
||||
public)
|
||||
MODE=public
|
||||
;;
|
||||
clean)
|
||||
MODE=clean
|
||||
;;
|
||||
help)
|
||||
MODE=help
|
||||
;;
|
||||
*)
|
||||
CMD="$CMD $i"
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
echo "Stable Diffusion server: $MODE"
|
||||
VER=$(git log -1 --pretty=format:"%h %ad")
|
||||
URL=$(git remote get-url origin)
|
||||
LSB=$(lsb_release -ds 2>/dev/null)
|
||||
UNAME=$(uname -rm 2>/dev/null)
|
||||
# SMI=$(nvidia-smi --query-gpu=name,driver_version --format=csv,noheader --id=0 2>/dev/null)
|
||||
echo "Version: $VER"
|
||||
echo "Repository: $URL"
|
||||
echo "Platform: $LSB $UNAME"
|
||||
|
||||
git-version () {
|
||||
pushd $1 >/dev/null
|
||||
BRANCH=$(git branch | grep -E 'main|master' | tail -1 | awk '{print $NF}')
|
||||
VER=$(git log -1 --pretty=format:"%h %ad")
|
||||
URL=$(git remote get-url origin)
|
||||
popd >/dev/null
|
||||
echo "- $VER $BRANCH $URL"
|
||||
}
|
||||
|
||||
git-update () {
|
||||
pushd $1 >/dev/null
|
||||
BRANCH=$(git branch | grep -E 'main|master' | tail -1 | awk '{print $NF}')
|
||||
git checkout --quiet $BRANCH
|
||||
git pull --quiet --rebase --autostash
|
||||
popd >/dev/null
|
||||
git-version $1
|
||||
}
|
||||
|
||||
if [ "$MODE" == update ]; then
|
||||
MERGE=$(git log --pretty=format:"%ad %s" | grep "Merge pull" | head -1)
|
||||
echo "Last Merge: $MERGE"
|
||||
|
||||
echo "Updating main repository"
|
||||
git-update .
|
||||
|
||||
echo "Installing general requirements"
|
||||
"$PYTHON" -m pip install --disable-pip-version-check --quiet --no-warn-conflicts --requirement requirements.txt
|
||||
|
||||
echo "Installing versioned requirements"
|
||||
"$PYTHON" -m pip install --disable-pip-version-check --quiet --no-warn-conflicts --requirement requirements_versions.txt
|
||||
|
||||
"$PYTHON" launch.py --exit
|
||||
echo "Local changes"
|
||||
git status --untracked=no --ignore-submodules=all --short
|
||||
echo "Note: To update any new dependencies or submodules, run 'automatic.sh install'"
|
||||
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$MODE" == help ]; then
|
||||
"$PYTHON" webui.py --help
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$MODE" == install ]; then
|
||||
MERGE=$(git log --pretty=format:"%ad %s" | grep "Merge pull" | head -1)
|
||||
echo "Last Merge: $MERGE"
|
||||
|
||||
"$PYTHON" -m pip --version
|
||||
"$PYTHON" -c 'import torch; import platform; print("- Python:", platform.python_version(), "Torch:", torch.__version__, "CUDA:", torch.version.cuda, "cuDNN:", torch.backends.cudnn.version(), "GPU:", torch.cuda.get_device_name(torch.cuda.current_device()), "Arch:", torch.cuda.get_device_capability());'
|
||||
|
||||
echo "Installing general requirements"
|
||||
"$PYTHON" -m pip install --disable-pip-version-check --quiet --no-warn-conflicts --requirement requirements.txt
|
||||
|
||||
echo "Installing versioned requirements"
|
||||
"$PYTHON" -m pip install --disable-pip-version-check --quiet --no-warn-conflicts --requirement requirements_versions.txt
|
||||
|
||||
echo "Updating submodules"
|
||||
git submodule --quiet update --init --recursive
|
||||
git submodule --quiet foreach 'echo $sm_path' | while read LINE; do git-update $LINE ; done
|
||||
# git submodule --quiet update --rebase --remote
|
||||
# git submodule foreach --quiet 'VER=$(git log -1 --pretty=format:"%h %ad"); BRANCH=$(git branch); URL=$(git remote get-url origin); echo "- $VER $BRANCH $URL"'
|
||||
|
||||
echo "Updating extensions"
|
||||
ls extensions/ | while read LINE; do git-update extensions/$LINE ; done
|
||||
|
||||
echo "Updating wiki"
|
||||
git-update wiki
|
||||
git-update wiki/origin-wiki
|
||||
|
||||
echo "Detached repos"
|
||||
ls repositories/ | while read LINE; do git-version repositories/$LINE ; done
|
||||
|
||||
"$PYTHON" launch.py --exit
|
||||
|
||||
echo "Local changes"
|
||||
git status --untracked=no --ignore-submodules=all --short
|
||||
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$MODE" == clean ]; then
|
||||
CMD="--disable-opt-split-attention"
|
||||
"$PYTHON" launch.py $CMD
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ $MODE == public ]; then
|
||||
CMD="$CMD --port 7860 --gradio-auth admin:pwd --listen --enable-insecure-extension-access"
|
||||
fi
|
||||
|
||||
if [ $MODE == optimized ]; then
|
||||
CMD="$CMD"
|
||||
fi
|
||||
|
||||
# exec accelerate launch --no_python --quiet --num_cpu_threads_per_process=6 "$PYTHON" $CMD
|
||||
exec "$PYTHON" $CMD
|
||||
|
||||
# export LD_PRELOAD=libtcmalloc.so
|
||||
# TORCH_CUDA_ARCH_LIST="8.6"
|
||||
# --opt-channelslast
|
||||
|
|
@ -0,0 +1 @@
|
|||
mediapipe
|
||||
|
|
@ -18,6 +18,7 @@ from util import Map
|
|||
from rich.pretty import install as pretty_install
|
||||
from rich.traceback import install as traceback_install
|
||||
from rich.console import Console
|
||||
|
||||
console = Console(log_time=True, log_time_format='%H:%M:%S-%f')
|
||||
pretty_install(console=console)
|
||||
traceback_install(console=console, extra_lines=1, width=console.width, word_wrap=False, indent_guides=False)
|
||||
|
|
|
|||
243
launch.py
243
launch.py
|
|
@ -1,87 +1,54 @@
|
|||
# this scripts installs necessary requirements and launches main program in webui.py
|
||||
import subprocess
|
||||
import os
|
||||
import sys
|
||||
import importlib.util
|
||||
import shlex
|
||||
import platform
|
||||
import json
|
||||
|
||||
import setup
|
||||
from modules import cmd_args
|
||||
from modules.paths_internal import script_path, extensions_dir
|
||||
from rich import print
|
||||
from modules.paths_internal import script_path
|
||||
|
||||
try:
|
||||
from rich import print
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
|
||||
commandline_args = os.environ.get('COMMANDLINE_ARGS', "")
|
||||
sys.argv += shlex.split(commandline_args)
|
||||
|
||||
args, _ = cmd_args.parser.parse_known_args()
|
||||
|
||||
python = sys.executable
|
||||
git = os.environ.get('GIT', "git")
|
||||
index_url = os.environ.get('INDEX_URL', "")
|
||||
stored_commit_hash = None
|
||||
skip_install = False
|
||||
dir_repos = "repositories"
|
||||
|
||||
if 'GRADIO_ANALYTICS_ENABLED' not in os.environ:
|
||||
os.environ['GRADIO_ANALYTICS_ENABLED'] = 'False'
|
||||
|
||||
|
||||
def check_python_version():
|
||||
is_windows = platform.system() == "Windows"
|
||||
major = sys.version_info.major
|
||||
minor = sys.version_info.minor
|
||||
micro = sys.version_info.micro
|
||||
|
||||
if is_windows:
|
||||
supported_minors = [10]
|
||||
else:
|
||||
supported_minors = [9, 10, 11]
|
||||
|
||||
if not (major == 3 and minor in supported_minors):
|
||||
import modules.errors
|
||||
|
||||
modules.errors.print_error_explanation(f"Incompatible Python version: {major}.{minor}.{micro} required 3.9-3.11")
|
||||
|
||||
|
||||
def commit_hash():
|
||||
global stored_commit_hash
|
||||
|
||||
if stored_commit_hash is not None:
|
||||
return stored_commit_hash
|
||||
|
||||
try:
|
||||
stored_commit_hash = run(f"{git} rev-parse HEAD").strip()
|
||||
except Exception:
|
||||
stored_commit_hash = "<none>"
|
||||
|
||||
return stored_commit_hash
|
||||
|
||||
|
||||
def run(command, desc=None, errdesc=None, custom_env=None, live=False):
|
||||
if desc is not None:
|
||||
print(desc)
|
||||
|
||||
if live:
|
||||
result = subprocess.run(command, shell=True, env=os.environ if custom_env is None else custom_env)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(f"""{errdesc or 'Error running command'}.
|
||||
Command: {command}
|
||||
Error code: {result.returncode}""")
|
||||
|
||||
return ""
|
||||
raise RuntimeError(f"""{errdesc or 'Error running command'} Command: {command} Error code: {result.returncode}""")
|
||||
return ''
|
||||
|
||||
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, env=os.environ if custom_env is None else custom_env)
|
||||
|
||||
if result.returncode != 0:
|
||||
|
||||
message = f"""{errdesc or 'Error running command'}.
|
||||
Command: {command}
|
||||
Error code: {result.returncode}
|
||||
stdout: {result.stdout.decode(encoding="utf8", errors="ignore") if len(result.stdout)>0 else '<empty>'}
|
||||
stderr: {result.stderr.decode(encoding="utf8", errors="ignore") if len(result.stderr)>0 else '<empty>'}
|
||||
"""
|
||||
raise RuntimeError(message)
|
||||
raise RuntimeError(f"""{errdesc or 'Error running command'} Command: {command} Error code: {result.returncode}
|
||||
stdout: {result.stdout.decode(encoding="utf8", errors="ignore") if len(result.stdout)>0 else ''}
|
||||
stderr: {result.stderr.decode(encoding="utf8", errors="ignore") if len(result.stderr)>0 else ''}
|
||||
""")
|
||||
|
||||
return result.stdout.decode(encoding="utf8", errors="ignore")
|
||||
|
||||
|
|
@ -96,7 +63,6 @@ def is_installed(package):
|
|||
spec = importlib.util.find_spec(package)
|
||||
except ModuleNotFoundError:
|
||||
return False
|
||||
|
||||
return spec is not None
|
||||
|
||||
|
||||
|
|
@ -105,207 +71,40 @@ def repo_dir(name):
|
|||
|
||||
|
||||
def run_python(code, desc=None, errdesc=None):
|
||||
return run(f'"{python}" -c "{code}"', desc, errdesc)
|
||||
return run(f'"{sys.executable}" -c "{code}"', desc, errdesc)
|
||||
|
||||
|
||||
def run_pip(args, desc=None):
|
||||
if skip_install:
|
||||
return
|
||||
|
||||
index_url_line = f' --index-url {index_url}' if index_url != '' else ''
|
||||
return run(f'"{python}" -m pip {args} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}")
|
||||
return run(f'"{sys.executable}" -m pip {args} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}")
|
||||
|
||||
|
||||
def check_run_python(code):
|
||||
return check_run(f'"{python}" -c "{code}"')
|
||||
return check_run(f'"{sys.executable}" -c "{code}"')
|
||||
|
||||
|
||||
def git_clone(url, dir, name, commithash=None):
|
||||
# TODO clone into temporary dir and move if successful
|
||||
|
||||
if os.path.exists(dir):
|
||||
if commithash is None:
|
||||
return
|
||||
|
||||
current_hash = run(f'"{git}" -C "{dir}" rev-parse HEAD', None, f"Couldn't determine {name}'s hash: {commithash}").strip()
|
||||
if current_hash == commithash:
|
||||
return
|
||||
|
||||
run(f'"{git}" -C "{dir}" fetch', f"Fetching updates for {name}...", f"Couldn't fetch {name}")
|
||||
run(f'"{git}" -C "{dir}" checkout {commithash}', f"Checking out commit for {name} with hash: {commithash}...", f"Couldn't checkout commit {commithash} for {name}")
|
||||
return
|
||||
|
||||
run(f'"{git}" clone "{url}" "{dir}"', f"Cloning {name} into {dir}...", f"Couldn't clone {name}")
|
||||
|
||||
if commithash is not None:
|
||||
run(f'"{git}" -C "{dir}" checkout {commithash}', None, "Couldn't checkout {name}'s hash: {commithash}")
|
||||
|
||||
|
||||
def git_pull_recursive(dir):
|
||||
for subdir, _, _ in os.walk(dir):
|
||||
if os.path.exists(os.path.join(subdir, '.git')):
|
||||
try:
|
||||
output = subprocess.check_output([git, '-C', subdir, 'pull', '--autostash'])
|
||||
print(f"Pulled changes for repository in '{subdir}':\n{output.decode('utf-8').strip()}\n")
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Couldn't perform 'git pull' on repository in '{subdir}':\n{e.output.decode('utf-8').strip()}\n")
|
||||
|
||||
|
||||
def version_check(commit):
|
||||
try:
|
||||
import requests
|
||||
commits = requests.get('https://api.github.com/repos/vladmandic/automatic/branches/master').json()
|
||||
if commit != "<none>" and commits['commit']['sha'] != commit:
|
||||
print("--------------------------------------------------------")
|
||||
print("| You are not up to date with the most recent release. |")
|
||||
print("| Consider running `git pull` to update. |")
|
||||
print("--------------------------------------------------------")
|
||||
elif commits['commit']['sha'] == commit:
|
||||
print("You are up to date with the most recent release.")
|
||||
else:
|
||||
print("Not a git clone, can't perform version check.")
|
||||
except Exception as e:
|
||||
print("version check failed", e)
|
||||
|
||||
|
||||
def run_extension_installer(extension_dir):
|
||||
path_installer = os.path.join(extension_dir, "install.py")
|
||||
if not os.path.isfile(path_installer):
|
||||
return
|
||||
|
||||
try:
|
||||
print('Running extension installer:', path_installer)
|
||||
env = os.environ.copy()
|
||||
env['PYTHONPATH'] = os.path.abspath(".")
|
||||
|
||||
stdout = run(f'"{python}" "{path_installer}"', errdesc=f"Error running install.py for extension {extension_dir}", custom_env=env)
|
||||
if stdout is not None and len(stdout) > 0:
|
||||
print('A', stdout)
|
||||
except Exception as e:
|
||||
print(e, file=sys.stderr)
|
||||
|
||||
|
||||
def list_extensions(settings_file):
|
||||
settings = {}
|
||||
|
||||
try:
|
||||
if os.path.isfile(settings_file):
|
||||
with open(settings_file, "r", encoding="utf8") as file:
|
||||
settings = json.load(file)
|
||||
except Exception as e:
|
||||
print(e, file=sys.stderr)
|
||||
|
||||
disabled_extensions = set(settings.get('disabled_extensions', []))
|
||||
disable_all_extensions = settings.get('disable_all_extensions', 'none')
|
||||
|
||||
if disable_all_extensions != 'none':
|
||||
return []
|
||||
|
||||
return [x for x in os.listdir(extensions_dir) if x not in disabled_extensions]
|
||||
|
||||
|
||||
def run_extensions_installers(settings_file):
|
||||
if not os.path.isdir(extensions_dir):
|
||||
return
|
||||
|
||||
for dirname_extension in list_extensions(settings_file):
|
||||
run_extension_installer(os.path.join(extensions_dir, dirname_extension))
|
||||
|
||||
|
||||
def prepare_environment():
|
||||
global skip_install
|
||||
|
||||
torch_command = os.environ.get('TORCH_COMMAND', "pip install torch torchaudio torchvision --extra-index-url https://download.pytorch.org/whl/cu118")
|
||||
requirements_file = os.environ.get('REQS_FILE', "requirements_versions.txt")
|
||||
|
||||
xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.18')
|
||||
gfpgan_package = os.environ.get('GFPGAN_PACKAGE', "git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379")
|
||||
clip_package = os.environ.get('CLIP_PACKAGE', "git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1")
|
||||
openclip_package = os.environ.get('OPENCLIP_PACKAGE', "git+https://github.com/mlfoundations/open_clip.git@bb6e834e9c70d9c27d0dc3ecedeebeaeb1ffad6b")
|
||||
|
||||
stable_diffusion_repo = os.environ.get('STABLE_DIFFUSION_REPO', "https://github.com/Stability-AI/stablediffusion.git")
|
||||
taming_transformers_repo = os.environ.get('TAMING_TRANSFORMERS_REPO', "https://github.com/CompVis/taming-transformers.git")
|
||||
k_diffusion_repo = os.environ.get('K_DIFFUSION_REPO', 'https://github.com/crowsonkb/k-diffusion.git')
|
||||
codeformer_repo = os.environ.get('CODEFORMER_REPO', 'https://github.com/sczhou/CodeFormer.git')
|
||||
blip_repo = os.environ.get('BLIP_REPO', 'https://github.com/salesforce/BLIP.git')
|
||||
|
||||
stable_diffusion_commit_hash = os.environ.get('STABLE_DIFFUSION_COMMIT_HASH', "cf1d67a6fd5ea1aa600c4df58e5b47da45f6bdbf")
|
||||
taming_transformers_commit_hash = os.environ.get('TAMING_TRANSFORMERS_COMMIT_HASH', "3ba01b241669f5ade541ce990f7650a3b8f65318")
|
||||
k_diffusion_commit_hash = os.environ.get('K_DIFFUSION_COMMIT_HASH', "b43db16749d51055f813255eea2fdf1def801919")
|
||||
codeformer_commit_hash = os.environ.get('CODEFORMER_COMMIT_HASH', "c5b4593074ba6214284d6acd5f1719b6c5d739af")
|
||||
blip_commit_hash = os.environ.get('BLIP_COMMIT_HASH', "48211a1594f1321b00f14c9f7a5b4813144b2fb9")
|
||||
|
||||
if not args.skip_python_version_check:
|
||||
check_python_version()
|
||||
|
||||
commit = commit_hash()
|
||||
|
||||
if args.reinstall_torch or not is_installed("torch") or not is_installed("torchvision"):
|
||||
run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True)
|
||||
|
||||
if not args.skip_torch_cuda_test:
|
||||
run_python("import torch; assert torch.cuda.is_available(), 'Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check'")
|
||||
|
||||
if not is_installed("gfpgan"):
|
||||
run_pip(f"install {gfpgan_package}", "gfpgan")
|
||||
|
||||
if not is_installed("clip"):
|
||||
run_pip(f"install {clip_package}", "clip")
|
||||
|
||||
if not is_installed("open_clip"):
|
||||
run_pip(f"install {openclip_package}", "open_clip")
|
||||
|
||||
if not is_installed("xformers") and args.reinstall_xformers and args.xformers:
|
||||
if platform.system() == "Windows":
|
||||
if platform.python_version().startswith("3.10"):
|
||||
run_pip(f"install -U -I --no-deps {xformers_package}", "xformers")
|
||||
else:
|
||||
print("Installation of xformers is not supported in this version of Python.")
|
||||
if not is_installed("xformers"):
|
||||
exit(0)
|
||||
elif platform.system() == "Linux":
|
||||
run_pip(f"install {xformers_package}", "xformers")
|
||||
|
||||
if not is_installed("pyngrok") and args.ngrok:
|
||||
run_pip("install pyngrok", "ngrok")
|
||||
|
||||
os.makedirs(os.path.join(script_path, dir_repos), exist_ok=True)
|
||||
|
||||
git_clone(stable_diffusion_repo, repo_dir('stable-diffusion-stability-ai'), "Stable Diffusion", stable_diffusion_commit_hash)
|
||||
git_clone(taming_transformers_repo, repo_dir('taming-transformers'), "Taming Transformers", taming_transformers_commit_hash)
|
||||
git_clone(k_diffusion_repo, repo_dir('k-diffusion'), "K-diffusion", k_diffusion_commit_hash)
|
||||
git_clone(codeformer_repo, repo_dir('CodeFormer'), "CodeFormer", codeformer_commit_hash)
|
||||
git_clone(blip_repo, repo_dir('BLIP'), "BLIP", blip_commit_hash)
|
||||
|
||||
if not is_installed("lpips"):
|
||||
run_pip(f"install -r \"{os.path.join(repo_dir('CodeFormer'), 'requirements.txt')}\"", "requirements for CodeFormer")
|
||||
|
||||
if not os.path.isfile(requirements_file):
|
||||
requirements_file = os.path.join(script_path, requirements_file)
|
||||
run_pip(f"install -r \"{requirements_file}\"", "requirements for Web UI")
|
||||
|
||||
run_extensions_installers(settings_file=args.ui_settings_file)
|
||||
|
||||
if args.update_check:
|
||||
version_check(commit)
|
||||
|
||||
if args.update_all_extensions:
|
||||
git_pull_recursive(extensions_dir)
|
||||
|
||||
if "--exit" in sys.argv:
|
||||
print("Exiting because of --exit argument")
|
||||
exit(0)
|
||||
|
||||
|
||||
def start():
|
||||
if __name__ == "__main__":
|
||||
setup.run_setup(False)
|
||||
setup.set_environment()
|
||||
# setup.check_torch()
|
||||
print(f"Launching {'API server' if '--nowebui' in sys.argv else 'Web UI'} with arguments: {' '.join(sys.argv[1:])}")
|
||||
import webui
|
||||
if '--nowebui' in sys.argv:
|
||||
webui.api_only()
|
||||
else:
|
||||
webui.webui()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
prepare_environment()
|
||||
start()
|
||||
|
|
|
|||
|
|
@ -5,15 +5,6 @@ from modules.paths_internal import models_path, script_path, data_path, extensio
|
|||
parser = argparse.ArgumentParser()
|
||||
|
||||
parser.add_argument("-f", action='store_true', help=argparse.SUPPRESS) # allows running as root; implemented outside of webui
|
||||
parser.add_argument("--update-all-extensions", action='store_true', help="launch.py argument: download updates for all extensions when starting the program")
|
||||
parser.add_argument("--skip-python-version-check", action='store_true', help="launch.py argument: do not check python version")
|
||||
parser.add_argument("--skip-torch-cuda-test", action='store_true', help="launch.py argument: do not check if CUDA is able to work properly")
|
||||
parser.add_argument("--reinstall-xformers", action='store_true', help="launch.py argument: install the appropriate version of xformers even if you have some version already installed")
|
||||
parser.add_argument("--reinstall-torch", action='store_true', help="launch.py argument: install the appropriate version of torch even if you have some version already installed")
|
||||
parser.add_argument("--update-check", action='store_true', help="launch.py argument: chck for updates at startup")
|
||||
parser.add_argument("--tests", type=str, default=None, help="launch.py argument: run tests in the specified directory")
|
||||
parser.add_argument("--no-tests", action='store_true', help="launch.py argument: do not run tests even if --tests option is specified")
|
||||
parser.add_argument("--skip-install", action='store_true', help="launch.py argument: skip installation of packages")
|
||||
parser.add_argument("--data-dir", type=str, default=os.path.dirname(os.path.dirname(os.path.realpath(__file__))), help="base path where all user data is stored")
|
||||
parser.add_argument("--config", type=str, default=sd_default_config, help="path to config which constructs model",)
|
||||
parser.add_argument("--ckpt", type=str, default=sd_model_file, help="path to checkpoint of stable diffusion model; if specified, this checkpoint will be added to the list of checkpoints and loaded",)
|
||||
|
|
@ -58,29 +49,23 @@ parser.add_argument("--sub-quad-kv-chunk-size", type=int, help="kv chunk size fo
|
|||
parser.add_argument("--sub-quad-chunk-threshold", type=int, help="the percentage of VRAM threshold for the sub-quadratic cross-attention layer optimization to use chunking", default=None)
|
||||
parser.add_argument("--opt-split-attention-invokeai", action='store_true', help="force-enables InvokeAI's cross-attention layer optimization. By default, it's on when cuda is unavailable.")
|
||||
parser.add_argument("--opt-split-attention-v1", action='store_true', help="enable older version of split attention optimization that does not consume all the VRAM it can find")
|
||||
parser.add_argument("--disable-sdp-attention", action='store_true', help="disable scaled dot product cross-attention layer optimization; requires PyTorch 2.*", default=False)
|
||||
parser.add_argument("--opt-sdp-attention", action='store_true', help="enable scaled dot product cross-attention layer optimization; requires PyTorch 2.*", default=True)
|
||||
parser.add_argument("--disable-sdp-attention", action='store_true', help="disable scaled dot product cross-attention layer optimization", default=False)
|
||||
parser.add_argument("--opt-sdp-attention", action='store_true', help="enable scaled dot product cross-attention layer optimization", default=True)
|
||||
parser.add_argument("--opt-sdp-no-mem-attention", action='store_true', help="enable scaled dot product cross-attention layer optimization without memory efficient attention, makes image generation deterministic; requires PyTorch 2.*")
|
||||
parser.add_argument("--disable-opt-split-attention", action='store_true', help="force-disables cross-attention layer optimization")
|
||||
parser.add_argument("--disable-nan-check", action='store_true', help="do not check if produced images/latent spaces have nans; useful for running without a checkpoint in CI", default=True)
|
||||
parser.add_argument("--use-cpu", nargs='+', help="use CPU as torch device for specified modules", default=[], type=str.lower)
|
||||
parser.add_argument("--listen", action='store_true', help="launch gradio with 0.0.0.0 as server name, allowing to respond to network requests")
|
||||
parser.add_argument("--port", type=int, help="launch gradio with given server port, you need root/admin rights for ports < 1024, defaults to 7860 if available", default=None)
|
||||
parser.add_argument("--show-negative-prompt", action='store_true', help="does not do anything", default=False)
|
||||
parser.add_argument("--ui-config-file", type=str, help="filename to use for ui configuration", default=os.path.join(data_path, 'ui-config.json'))
|
||||
parser.add_argument("--hide-ui-dir-config", action='store_true', help="hide directory configuration from webui", default=False)
|
||||
parser.add_argument("--freeze-settings", action='store_true', help="disable editing settings", default=False)
|
||||
parser.add_argument("--ui-settings-file", type=str, help="filename to use for ui settings", default=os.path.join(data_path, 'config.json'))
|
||||
parser.add_argument("--gradio-debug", action='store_true', help="launch gradio with --debug option")
|
||||
parser.add_argument("--gradio-auth", type=str, help='set gradio authentication like "username:password"; or comma-delimit multiple like "u1:p1,u2:p2,u3:p3"', default=None)
|
||||
parser.add_argument("--gradio-auth-path", type=str, help='set gradio authentication file path ex. "/path/to/auth/file" same auth format as --gradio-auth', default=None)
|
||||
parser.add_argument("--gradio-img2img-tool", type=str, help='does not do anything')
|
||||
parser.add_argument("--gradio-inpaint-tool", type=str, help="does not do anything")
|
||||
parser.add_argument("--opt-channelslast", action='store_true', help="change memory type for stable diffusion to channels last")
|
||||
parser.add_argument("--styles-file", type=str, help="filename to use for styles", default=os.path.join(data_path, 'styles.csv'))
|
||||
parser.add_argument("--autolaunch", action='store_true', help="open the webui URL in the system's default browser upon launch", default=False)
|
||||
parser.add_argument("--theme", type=str, help="launches the UI with light or dark theme", default=None)
|
||||
parser.add_argument("--use-textbox-seed", action='store_true', help="use textbox for seeds in UI (no up/down, but possible to input long seeds)", default=False)
|
||||
parser.add_argument("--disable-console-progressbars", action='store_true', help="do not output progressbars to console", default=True)
|
||||
parser.add_argument("--enable-console-prompts", action='store_true', help="print prompts to console when generating with txt2img and img2img", default=False)
|
||||
parser.add_argument('--vae-path', type=str, help='Checkpoint to use as VAE; setting this argument disables all settings related to VAE', default=None)
|
||||
|
|
@ -88,8 +73,6 @@ parser.add_argument("--disable-safe-unpickle", action='store_true', help="disabl
|
|||
parser.add_argument("--api", action='store_true', help="use api=True to launch the API together with the webui (use --nowebui instead for only the API)", default=True)
|
||||
parser.add_argument("--api-auth", type=str, help='Set authentication for API like "username:password"; or comma-delimit multiple like "u1:p1,u2:p2,u3:p3"', default=None)
|
||||
parser.add_argument("--api-log", action='store_true', help="use api-log=True to enable logging of all API requests")
|
||||
parser.add_argument("--nowebui", action='store_true', help="use api=True to launch the API instead of the webui")
|
||||
parser.add_argument("--ui-debug-mode", action='store_true', help="Don't load model to quickly launch UI")
|
||||
parser.add_argument("--device-id", type=str, help="Select the default CUDA device to use (export CUDA_VISIBLE_DEVICES=0,1,etc might be needed before)", default=None)
|
||||
parser.add_argument("--administrator", action='store_true', help="Administrator rights", default=False)
|
||||
parser.add_argument("--cors-allow-origins", type=str, help="Allowed CORS origin(s) in the form of a comma-separated list (no spaces)", default=None)
|
||||
|
|
@ -97,9 +80,12 @@ parser.add_argument("--cors-allow-origins-regex", type=str, help="Allowed CORS o
|
|||
parser.add_argument("--tls-keyfile", type=str, help="Partially enables TLS, requires --tls-certfile to fully function", default=None)
|
||||
parser.add_argument("--tls-certfile", type=str, help="Partially enables TLS, requires --tls-keyfile to fully function", default=None)
|
||||
parser.add_argument("--server-name", type=str, help="Sets hostname of server", default=None)
|
||||
parser.add_argument("--gradio-queue", action='store_true', help="does not do anything", default=True)
|
||||
parser.add_argument("--no-gradio-queue", action='store_true', help="Disables gradio queue; causes the webpage to use http requests instead of websockets; was the defaul in earlier versions")
|
||||
parser.add_argument("--skip-version-check", action='store_true', help="Do not check versions of torch and xformers", default=True)
|
||||
parser.add_argument("--no-hashing", action='store_true', help="disable sha256 hashing of checkpoints to help loading performance", default=False)
|
||||
parser.add_argument("--no-download-sd-model", action='store_true', help="don't download SD1.5 model even if no model is found in --ckpt-dir", default=False)
|
||||
parser.add_argument("--profile", action='store_true', help="run profiler")
|
||||
|
||||
# used by setup.py
|
||||
parser.add_argument('--quick', default = False, action='store_true', help = "Skip installing if setup.log is newer than repo timestamp, default: %(default)s")
|
||||
parser.add_argument('--upgrade', default = False, action='store_true', help = "Upgrade main repository to latest version, default: %(default)s")
|
||||
parser.add_argument('--update', default = False, action='store_true', help = "Update all extensions and submodules, default: %(default)s")
|
||||
parser.add_argument('--skip-extensions', default = False, action='store_true', help = "Skips running individual extension installers, default: %(default)s")
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
accelerate
|
||||
addict
|
||||
aenum
|
||||
aiohttp
|
||||
anyio
|
||||
|
|
@ -8,45 +8,60 @@ basicsr
|
|||
bitsandbytes
|
||||
blendmodes
|
||||
clean-fid
|
||||
clip-interrogator
|
||||
colormap
|
||||
diffusers
|
||||
easydev
|
||||
einops
|
||||
extcolors
|
||||
facexlib
|
||||
fastapi
|
||||
filetype
|
||||
font-roboto
|
||||
fonts
|
||||
gfpgan
|
||||
GitPython
|
||||
gradio
|
||||
future
|
||||
gdown
|
||||
httpcore
|
||||
inflection
|
||||
invisible-watermark
|
||||
jsonmerge
|
||||
kornia
|
||||
lark
|
||||
mediapipe
|
||||
lmdb
|
||||
lpips
|
||||
numpy
|
||||
omegaconf
|
||||
opencv-contrib-python
|
||||
opencv-python
|
||||
piexif
|
||||
Pillow
|
||||
psutil
|
||||
pyngrok
|
||||
pytorch_lightning
|
||||
realesrgan
|
||||
pyyaml
|
||||
requests
|
||||
resize-right
|
||||
rich
|
||||
safetensors
|
||||
scikit-image
|
||||
scipy
|
||||
tb_nightly
|
||||
timm
|
||||
toml
|
||||
torch
|
||||
torchdiffeq
|
||||
torchsde
|
||||
transformers
|
||||
torchvision
|
||||
tqdm
|
||||
voluptuous
|
||||
yapf
|
||||
|
||||
accelerate==0.18.0
|
||||
diffusers==0.14.0
|
||||
einops==0.4.1
|
||||
fastapi==0.94.0
|
||||
gfpgan==1.3.8
|
||||
GitPython==3.1.31
|
||||
gradio==3.23.0
|
||||
numexpr==2.8.4
|
||||
omegaconf==2.3.0
|
||||
pandas==1.5.3
|
||||
protobuf==3.20.3
|
||||
pytorch_lightning==1.9.4
|
||||
realesrgan==0.3.0
|
||||
safetensors==0.3.0
|
||||
tensorflow==2.12.0
|
||||
transformers==4.27.4
|
||||
|
|
|
|||
|
|
@ -1,18 +0,0 @@
|
|||
accelerate==0.18.0
|
||||
clip_interrogator==0.6.0
|
||||
diffusers==0.14.0
|
||||
einops==0.4.1
|
||||
fastapi==0.94.0
|
||||
gfpgan==1.3.8
|
||||
GitPython==3.1.31
|
||||
gradio==3.23
|
||||
numexpr==2.8.4
|
||||
omegaconf==2.3.0
|
||||
pandas==1.5.3
|
||||
Pillow==9.4.0
|
||||
protobuf==3.20.3
|
||||
pytorch_lightning==1.9.4
|
||||
realesrgan==0.3.0
|
||||
safetensors==0.3.0
|
||||
tensorflow==2.12.0
|
||||
transformers==4.27.4
|
||||
|
|
@ -0,0 +1,325 @@
|
|||
import os
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
import subprocess
|
||||
import argparse
|
||||
|
||||
|
||||
import logging
|
||||
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s | %(levelname)s | %(message)s', filename='setup.log', filemode='w')
|
||||
log = logging.getLogger("sd-setup")
|
||||
try:
|
||||
from rich.logging import RichHandler
|
||||
from rich.console import Console
|
||||
from rich.pretty import install as pretty_install
|
||||
from rich.traceback import install as traceback_install
|
||||
rh = RichHandler(show_time=True, omit_repeated_times=False, show_level=True, show_path=True, markup=True, rich_tracebacks=True, log_time_format='%H:%M:%S-%f')
|
||||
rh.setLevel(logging.INFO)
|
||||
log.addHandler(rh)
|
||||
# logging.basicConfig(level=logging.INFO, format='%(message)s', handlers=[RichHandler(show_time=True, omit_repeated_times=False, show_level=True, show_path=True, markup=True, rich_tracebacks=True, log_time_format='%H:%M:%S-%f')])
|
||||
console = Console(log_time=True, log_time_format='%H:%M:%S-%f')
|
||||
pretty_install(console=console)
|
||||
traceback_install(console=console, extra_lines=1, width=console.width, word_wrap=False, indent_guides=False, suppress=[])
|
||||
except:
|
||||
pass
|
||||
sh = logging.StreamHandler()
|
||||
sh.setLevel(logging.INFO)
|
||||
log.addHandler(sh)
|
||||
|
||||
parser = argparse.ArgumentParser(description = 'Setup for SD WebUI')
|
||||
parser.add_argument('--quick', default = False, action='store_true', help = "Skip installing if setup.log is newer than repo timestamp, default: %(default)s")
|
||||
parser.add_argument('--upgrade', default = False, action='store_true', help = "Upgrade main repository to latest version, default: %(default)s")
|
||||
parser.add_argument('--update', default = False, action='store_true', help = "Update all extensions and submodules, default: %(default)s")
|
||||
parser.add_argument('--skip-extensions', default = False, action='store_true', help = "Skips running individual extension installers, default: %(default)s")
|
||||
args = parser.parse_args()
|
||||
|
||||
|
||||
def install(package):
|
||||
def pip(args: str):
|
||||
log.debug(f"Running pip: {args}")
|
||||
result = subprocess.run(f'"{sys.executable}" -m pip {args}', shell=True, env=os.environ, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
txt = result.stdout.decode(encoding="utf8", errors="ignore")
|
||||
if len(result.stderr) > 0:
|
||||
txt = txt + '\n' + result.stderr.decode(encoding="utf8", errors="ignore")
|
||||
if result.returncode != 0:
|
||||
log.error(f'Error running pip with args: {args}')
|
||||
log.debug(f'Pip output: {txt}')
|
||||
return txt
|
||||
|
||||
def installed():
|
||||
import pkg_resources
|
||||
ok = True
|
||||
try:
|
||||
pkgs = [p for p in package.split() if not p.startswith('-') and not p.startswith('git+') and not p.startswith('http') and not p.startswith('=')]
|
||||
for pkg in pkgs:
|
||||
p = pkg.split('==')
|
||||
spec = pkg_resources.working_set.by_key.get(p[0], None)
|
||||
if spec is None:
|
||||
spec = pkg_resources.working_set.by_key.get(p[0].lower(), None)
|
||||
if spec is None:
|
||||
spec = pkg_resources.working_set.by_key.get(p[0].replace('_', '-'), None)
|
||||
ok = ok and spec is not None
|
||||
version = pkg_resources.get_distribution(p[0]).version
|
||||
if ok and len(p) > 1:
|
||||
ok = ok and version == p[1]
|
||||
if not ok:
|
||||
log.warning(f"Package wrong version found: {p[0]} {version} required {p[1]}")
|
||||
# if ok:
|
||||
# log.debug(f"Package already installed: {p[0]} {version}")
|
||||
# else:
|
||||
# log.debug(f"Package not installed: {p[0]} {version}")
|
||||
return ok
|
||||
except ModuleNotFoundError:
|
||||
log.debug(f"Package not installed: {pkgs}")
|
||||
return False
|
||||
|
||||
if not installed():
|
||||
pip(f"install --upgrade {package}")
|
||||
|
||||
|
||||
def git(args: str):
|
||||
# log.debug(f"Running git: {args}")
|
||||
git_cmd = os.environ.get('GIT', "git")
|
||||
result = subprocess.run(f'"{git_cmd}" {args}', shell=True, env=os.environ, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
txt = result.stdout.decode(encoding="utf8", errors="ignore")
|
||||
if len(result.stderr) > 0:
|
||||
txt = txt + '\n' + result.stderr.decode(encoding="utf8", errors="ignore")
|
||||
if result.returncode != 0:
|
||||
log.error(f'Error running git with args: {args}')
|
||||
log.debug(f'Git output: {txt}')
|
||||
return txt
|
||||
|
||||
|
||||
def update(dir):
|
||||
branch = git(f'-C "{dir}" branch')
|
||||
if 'main' in branch:
|
||||
# log.debug(f'Using main branch {dir}')
|
||||
git(f'-C "{dir}" checkout main')
|
||||
elif 'master' in branch:
|
||||
# log.debug(f'Using master branch {dir}')
|
||||
git(f'-C "{dir}" checkout master')
|
||||
else:
|
||||
log.warning(f'Unknown branch for: {dir}')
|
||||
git(f'-C "{dir}" pull --rebase --autostash')
|
||||
|
||||
|
||||
def clone(url, dir, commithash=None):
|
||||
|
||||
if os.path.exists(dir):
|
||||
if commithash is None:
|
||||
return
|
||||
current_hash = git(f'-C "{dir}" rev-parse HEAD').strip()
|
||||
if current_hash != commithash:
|
||||
git(f'-C "{dir}" fetch')
|
||||
git(f'-C "{dir}" checkout {commithash}')
|
||||
return
|
||||
else:
|
||||
git(f'clone "{url}" "{dir}"')
|
||||
if commithash is not None:
|
||||
git(f'-C "{dir}" checkout {commithash}')
|
||||
|
||||
|
||||
def parse_env():
|
||||
import shlex
|
||||
args = os.environ.get('COMMANDLINE_ARGS', "")
|
||||
sys.argv += shlex.split(args)
|
||||
|
||||
|
||||
def check_python():
|
||||
import platform
|
||||
supported_minors = [10] if platform.system() != "Windows" else [9, 10, 11]
|
||||
log.info(f'Python {platform.python_version()} on {platform.system()}')
|
||||
if not (sys.version_info.major == 3 and sys.version_info.minor in supported_minors):
|
||||
raise RuntimeError(f"Incompatible Python version: {sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} required 3.9-3.11")
|
||||
|
||||
|
||||
def check_torch():
|
||||
install(f'torch torchaudio torchvision --extra-index-url https://download.pytorch.org/whl/cu118')
|
||||
try:
|
||||
import torch;
|
||||
log.info(f'Torch {torch.__version__}')
|
||||
if not torch.cuda.is_available():
|
||||
log.warning("Torch repoorts CUDA not available")
|
||||
else:
|
||||
log.info(f'Torch detected GPU: {torch.cuda.get_device_name(torch.cuda.current_device())}')
|
||||
if torch.version.cuda:
|
||||
log.info(f'Torch backend: nVidia CUDA {torch.version.cuda} cuDNN {torch.backends.cudnn.version()}')
|
||||
elif torch.version.hip:
|
||||
log.info(f'Torch backend: AMD ROCm HIP {torch.version.hip}')
|
||||
else:
|
||||
log.warning(f'Unknown Torch backend')
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def install_packages():
|
||||
log.info('Installing packages')
|
||||
gfpgan_package = os.environ.get('GFPGAN_PACKAGE', "git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379")
|
||||
clip_package = os.environ.get('CLIP_PACKAGE', "git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1")
|
||||
openclip_package = os.environ.get('OPENCLIP_PACKAGE', "git+https://github.com/mlfoundations/open_clip.git@bb6e834e9c70d9c27d0dc3ecedeebeaeb1ffad6b")
|
||||
xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.18')
|
||||
install(gfpgan_package)
|
||||
install(clip_package)
|
||||
install(openclip_package)
|
||||
install(f'--no-deps {xformers_package}')
|
||||
|
||||
|
||||
def install_repositories():
|
||||
def dir(name):
|
||||
return os.path.join(os.path.dirname(__file__), 'repositories', name)
|
||||
|
||||
log.info('Installing repositories')
|
||||
os.makedirs(os.path.join(os.path.dirname(__file__), 'repositories'), exist_ok=True)
|
||||
stable_diffusion_repo = os.environ.get('STABLE_DIFFUSION_REPO', "https://github.com/Stability-AI/stablediffusion.git")
|
||||
stable_diffusion_commit = os.environ.get('STABLE_DIFFUSION_COMMIT_HASH', "cf1d67a6fd5ea1aa600c4df58e5b47da45f6bdbf")
|
||||
clone(stable_diffusion_repo, dir('stable-diffusion-stability-ai'), stable_diffusion_commit)
|
||||
taming_transformers_repo = os.environ.get('TAMING_TRANSFORMERS_REPO', "https://github.com/CompVis/taming-transformers.git")
|
||||
taming_transformers_commit = os.environ.get('TAMING_TRANSFORMERS_COMMIT_HASH', "3ba01b241669f5ade541ce990f7650a3b8f65318")
|
||||
clone(taming_transformers_repo, dir('taming-transformers'), taming_transformers_commit)
|
||||
k_diffusion_repo = os.environ.get('K_DIFFUSION_REPO', 'https://github.com/crowsonkb/k-diffusion.git')
|
||||
k_diffusion_commit = os.environ.get('K_DIFFUSION_COMMIT_HASH', "b43db16749d51055f813255eea2fdf1def801919")
|
||||
clone(k_diffusion_repo, dir('k-diffusion'), k_diffusion_commit)
|
||||
codeformer_repo = os.environ.get('CODEFORMER_REPO', 'https://github.com/sczhou/CodeFormer.git')
|
||||
codeformer_commit = os.environ.get('CODEFORMER_COMMIT_HASH', "c5b4593074ba6214284d6acd5f1719b6c5d739af")
|
||||
clone(codeformer_repo, dir('CodeFormer'), codeformer_commit)
|
||||
blip_repo = os.environ.get('BLIP_REPO', 'https://github.com/salesforce/BLIP.git')
|
||||
blip_commit = os.environ.get('BLIP_COMMIT_HASH', "48211a1594f1321b00f14c9f7a5b4813144b2fb9")
|
||||
clone(blip_repo, dir('BLIP'), blip_commit)
|
||||
|
||||
|
||||
def install_extensions():
|
||||
def list_extensions(dir):
|
||||
settings = {}
|
||||
if os.path.isfile('config.json'):
|
||||
with open('config.json', "r", encoding="utf8") as file:
|
||||
settings = json.load(file)
|
||||
if settings.get('disable_all_extensions', 'none') != 'none':
|
||||
log.debug(f'Disabled extensions: all')
|
||||
return []
|
||||
else:
|
||||
disabled_extensions = set(settings.get('disabled_extensions', []))
|
||||
if len(disabled_extensions) > 0:
|
||||
log.debug(f'Disabled extensions: {disabled_extensions}')
|
||||
return [x for x in os.listdir(dir) if x not in disabled_extensions and not x.startswith('.')]
|
||||
|
||||
def run_extension_installer(extension_dir):
|
||||
path_installer = os.path.join(extension_dir, "install.py")
|
||||
if not os.path.isfile(path_installer):
|
||||
return
|
||||
try:
|
||||
log.debug(f"Running extension installer: {path_installer}")
|
||||
env = os.environ.copy()
|
||||
env['PYTHONPATH'] = os.path.abspath(".")
|
||||
result = subprocess.run(f'"{sys.executable}" "{path_installer}"', shell=True, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
if result.returncode != 0:
|
||||
txt = result.stdout.decode(encoding="utf8", errors="ignore")
|
||||
if len(result.stderr) > 0:
|
||||
txt = txt + '\n' + result.stderr.decode(encoding="utf8", errors="ignore")
|
||||
log.error(f'Error running extension installer: {txt}')
|
||||
except Exception as e:
|
||||
log.error(f'Exception running extension installer: {e}')
|
||||
|
||||
|
||||
extensions_builtin_dir = os.path.join(os.path.dirname(__file__), 'extensions-builtin')
|
||||
extensions = list_extensions(extensions_builtin_dir)
|
||||
log.info(f'Built-in extensions: {extensions}')
|
||||
for ext in extensions:
|
||||
if args.update:
|
||||
update(os.path.join(extensions_builtin_dir, ext))
|
||||
if not args.skip_extensions:
|
||||
run_extension_installer(os.path.join(extensions_builtin_dir, ext))
|
||||
|
||||
extensions_dir = os.path.join(os.path.dirname(__file__), 'extensions')
|
||||
extensions = list_extensions(extensions_dir)
|
||||
log.info(f'Enabled extensions: {extensions}')
|
||||
for ext in extensions:
|
||||
if args.update:
|
||||
update(os.path.join(extensions_dir, ext))
|
||||
if not args.skip_extensions:
|
||||
run_extension_installer(os.path.join(extensions_dir, ext))
|
||||
|
||||
|
||||
def install_submodules():
|
||||
log.info('Installing submodules')
|
||||
git(f'submodule --quiet update --init --recursive')
|
||||
if args.update:
|
||||
log.info('Updating submodules')
|
||||
submodules = git('submodule').splitlines()
|
||||
for submodule in submodules:
|
||||
name = submodule.split()[1].strip()
|
||||
update(name)
|
||||
|
||||
|
||||
def install_requirements():
|
||||
log.info('Installing requirements')
|
||||
f = open('requirements.txt', 'r')
|
||||
lines = [line.strip() for line in f.readlines() if line.strip() != '']
|
||||
for line in lines:
|
||||
install(line)
|
||||
|
||||
|
||||
def set_environment():
|
||||
log.info('Setting environment tuning')
|
||||
os.environ.setdefault('TF_CPP_MIN_LOG_LEVEL', '2')
|
||||
os.environ.setdefault('ACCELERATE', 'True')
|
||||
os.environ.setdefault('FORCE_CUDA', '1')
|
||||
os.environ.setdefault('ATTN_PRECISION', 'fp16')
|
||||
os.environ.setdefault('PYTORCH_CUDA_ALLOC_CONF', 'garbage_collection_threshold:0.9,max_split_size_mb:512')
|
||||
os.environ.setdefault('CUDA_LAUNCH_BLOCKING', '0')
|
||||
os.environ.setdefault('CUDA_CACHE_DISABLE', '0')
|
||||
os.environ.setdefault('CUDA_AUTO_BOOST', '1')
|
||||
os.environ.setdefault('CUDA_MODULE_LOADING', 'LAZY')
|
||||
os.environ.setdefault('CUDA_DEVICE_DEFAULT_PERSISTING_L2_CACHE_PERCENTAGE_LIMIT', '0')
|
||||
os.environ.setdefault('GRADIO_ANALYTICS_ENABLED', 'False')
|
||||
os.environ.setdefault('SAFETENSORS_FAST_GPU', '1')
|
||||
|
||||
|
||||
def check_version():
|
||||
ver = git('log -1 --pretty=format:"%h %ad"')
|
||||
log.info(f'Version: {ver}')
|
||||
hash = git('rev-parse HEAD')
|
||||
import requests
|
||||
logging.getLogger("urllib3").setLevel(logging.WARNING)
|
||||
commits = requests.get('https://api.github.com/repos/vladmandic/automatic/branches/master').json()
|
||||
if commits['commit']['sha'] != hash:
|
||||
if args.upgrade:
|
||||
update('.')
|
||||
ver = git('log -1 --pretty=format:"%h %ad"')
|
||||
log.info(f'Updated to version: {ver}')
|
||||
else:
|
||||
log.warning(f'Latest available version: {commits["commit"]["commit"]["author"]["date"]}')
|
||||
if args.update:
|
||||
log.info('Updating Wiki')
|
||||
update(os.path.join(os.path.dirname(__file__), "wiki"))
|
||||
update(os.path.join(os.path.dirname(__file__), "wiki", "origin-wiki"))
|
||||
|
||||
|
||||
def check_timestamp():
|
||||
if not os.path.isfile('setup.log'):
|
||||
return False
|
||||
setup_time = os.path.getmtime('setup.log')
|
||||
log.debug(f'Previous setup time: {time.ctime(setup_time)}')
|
||||
version_time = int(git('log -1 --pretty=format:"%at"'))
|
||||
log.debug(f'Repository update time: {time.ctime(int(version_time))}')
|
||||
return setup_time >= version_time
|
||||
|
||||
def run_setup(quick = False):
|
||||
parse_env()
|
||||
check_python()
|
||||
if (quick or args.quick) and check_timestamp():
|
||||
log.info('Attempting quick setup')
|
||||
return
|
||||
log.info("Running setup")
|
||||
install_requirements()
|
||||
check_version()
|
||||
install_packages()
|
||||
install_repositories()
|
||||
install_submodules()
|
||||
install_extensions()
|
||||
install_requirements()
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_setup()
|
||||
set_environment()
|
||||
check_torch()
|
||||
Loading…
Reference in New Issue