ui&bugs fix
Jaylen Lee
parent
1055122bde
commit
8b0763a228
23
README.md
23
README.md
|
|
@ -18,7 +18,7 @@ If you like the project, please give me a star! ⭐
|
|||
|
||||
The extension enables **large image drawing & upscaling with limited VRAM** via the following techniques:
|
||||
|
||||
1. Two SOTA diffusion tiling algorithms: [Mixture of Diffusers](https://github.com/albarji/mixture-of-diffusers) and [MultiDiffusion](https://multidiffusion.github.io), add [Demofusion](https://github.com/PRIS-CV/DemoFusion)
|
||||
1. SOTA diffusion tiling algorithms: [Mixture of Diffusers](https://github.com/albarji/mixture-of-diffusers) and [MultiDiffusion](https://multidiffusion.github.io), add [Demofusion](https://github.com/PRIS-CV/DemoFusion)
|
||||
2. My original Tiled VAE algorithm.
|
||||
3. My original TIled Noise Inversion for better upscaling.
|
||||
|
||||
|
|
@ -32,6 +32,7 @@ The extension enables **large image drawing & upscaling with limited VRAM** via
|
|||
- [x] [Regional Prompt Control](#region-prompt-control)
|
||||
- [x] [Img2img upscale](#img2img-upscale)
|
||||
- [x] [Ultra-Large image generation](#ultra-large-image-generation)
|
||||
- [x] [Demofusion available](#Demofusion available)
|
||||
|
||||
=> Quickstart Tutorial: [Tutorial for multidiffusion upscaler for automatic1111](https://civitai.com/models/34726), thanks to [@PotatoBananaApple](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/discussions/120) 🎉
|
||||
|
||||
|
|
@ -186,6 +187,26 @@ The extension enables **large image drawing & upscaling with limited VRAM** via
|
|||
|
||||
****
|
||||
|
||||
### Demofusion available
|
||||
|
||||
ℹ Please set seed in for a better result.
|
||||
|
||||
ℹ Ticking the random jitter usually yields better results.
|
||||
|
||||
ℹ Recommend using higher steps, such as 30 or more, for better results
|
||||
|
||||
ℹ If you set the image size to 512 * 512, the appropriate window size and overlap are 64 and 32 or smaller. If it is 1024, it is recommended to double it.
|
||||
|
||||
ℹ Recommend using a higher denoising strength in img2img，and try to use the original model, seeds, and prompt as much as possible
|
||||
|
||||
ℹ Do not enable it together with tilediffusion. It supports operations such as tilevae, noise inversion, etc.
|
||||
|
||||
ℹFor parameters such as c1, c2, and c3, please refer to the demofusion. You don't have to adjust it.
|
||||
|
||||
****
|
||||
|
||||
###
|
||||
|
||||
## Installation
|
||||
|
||||
⚪ Method 1: Official Market
|
||||
|
|
|
|||
23
README_CN.md
23
README_CN.md
|
|
@ -1,4 +1,4 @@
|
|||
# 用 Tiled Diffusion & VAE 生成大型图像
|
||||
用 Tiled Diffusion & VAE 生成大型图像
|
||||
|
||||
[![CC 署名-非商用-相同方式共享 4.0][cc-by-nc-sa-shield]][cc-by-nc-sa]
|
||||
|
||||
|
|
@ -28,6 +28,7 @@
|
|||
- [x] [区域提示控制](#区域提示控制)
|
||||
- [x] [Img2img 放大](#img2img-放大)
|
||||
- [x] [生成超大图像](#生成超大图像)
|
||||
- [x] [Demofusion现已可用](#Demofusion现已可用)
|
||||
|
||||
=> 快速入门教程: [Tutorial for multidiffusion upscaler for automatic1111](https://civitai.com/models/34726), 感谢由 [@PotatoBananaApple](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/discussions/120) 提供 🎉
|
||||
|
||||
|
|
@ -181,6 +182,26 @@
|
|||
|
||||
****
|
||||
|
||||
### Demofusion现已可用
|
||||
|
||||
ℹ 建议设定随机种子以获取更好的图像
|
||||
|
||||
ℹ 勾选随机抖动通常会得到更好的结果
|
||||
|
||||
ℹ推荐使用较高的步数，例如30步以上，往往会有更好的效果
|
||||
|
||||
ℹ如果你设定的生图大小是512*512， 合适的window size和overlap是64和32或者更小。如果是1024则推荐翻倍
|
||||
|
||||
ℹimg2img推荐使用较高的重绘幅度，并尽可能地使用原图的生图模型、随机种子以及prompt等
|
||||
|
||||
ℹ不要同时开启tilediffusion. 但该组件支持tilevae、noise inversion等常用功能
|
||||
|
||||
ℹc1，c2，c3等参数可以参考demofusion. 你不必调整这些参数
|
||||
|
||||
****
|
||||
|
||||
###
|
||||
|
||||
## 安装
|
||||
|
||||
⚪ 方法 1: 官方市场
|
||||
|
|
|
|||
|
|
@ -42,54 +42,45 @@ class Script(scripts.Script):
|
|||
with gr.Accordion('DemoFusion', open=False, elem_id=f'MD-{tab}'):
|
||||
with gr.Row(variant='compact') as tab_enable:
|
||||
enabled = gr.Checkbox(label='Enable DemoFusion(Do not open it with tilediffusion)', value=False, elem_id=uid('enabled'))
|
||||
# overwrite_size = gr.Checkbox(label='Overwrite image size', value=False, visible=not is_img2img, elem_id=uid('overwrite-image-size'))
|
||||
keep_input_size = gr.Checkbox(label='Keep input image size', value=True, visible=is_img2img, elem_id=uid('keep-input-size'))
|
||||
|
||||
with gr.Row(variant='compact') as tab_default:
|
||||
random_jitter = gr.Checkbox(label='Random jitter windows', value=True, elem_id=uid('random-jitter'))
|
||||
keep_input_size = gr.Checkbox(label='Keep input image size', value=True, visible=is_img2img, elem_id=uid('keep-input-size'))
|
||||
gaussian_filter = gr.Checkbox(label='Gaussian filter', value=False, elem_id=uid('gaussian'))
|
||||
|
||||
# with gr.Row(variant='compact', visible=False) as tab_size:
|
||||
# image_width = gr.Slider(minimum=256, maximum=16384, step=16, label='Image width', value=1024, elem_id=f'MD-overwrite-width-{tab}')
|
||||
# image_height = gr.Slider(minimum=256, maximum=16384, step=16, label='Image height', value=1024, elem_id=f'MD-overwrite-height-{tab}')
|
||||
# overwrite_size.change(fn=gr_show, inputs=overwrite_size, outputs=tab_size, show_progress=False)
|
||||
|
||||
# with gr.Row(variant='compact', visible=True) as tab_size:
|
||||
# c1 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c1', value=3, elem_id=f'c1-{tab}')
|
||||
# c2 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c2', value=1, elem_id=f'c2-{tab}')
|
||||
# c3 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c3', value=1, elem_id=f'c3-{tab}')
|
||||
|
||||
with gr.Row(variant='compact') as tab_param:
|
||||
method = gr.Dropdown(label='Method', choices=[Method_2.DEMO_FU.value], value=Method_2.DEMO_FU.value, elem_id=uid('method-2'))
|
||||
control_tensor_cpu = gr.Checkbox(label='Move ControlNet tensor to CPU (if applicable)', value=False, elem_id=uid('control-tensor-cpu-2'))
|
||||
method = gr.Dropdown(label='Method', choices=[Method_2.DEMO_FU.value], value=Method_2.DEMO_FU.value, elem_id=uid('method'))
|
||||
control_tensor_cpu = gr.Checkbox(label='Move ControlNet tensor to CPU (if applicable)', value=False, elem_id=uid('control-tensor-cpu'))
|
||||
reset_status = gr.Button(value='Free GPU', variant='tool')
|
||||
reset_status.click(fn=self.reset_and_gc, show_progress=False)
|
||||
|
||||
with gr.Group() as tab_tile:
|
||||
with gr.Row(variant='compact'):
|
||||
window_size = gr.Slider(minimum=16, maximum=256, step=16, label='Latent window size', value=128, elem_id=uid('latent-window-size'))
|
||||
# tile_height = gr.Slider(minimum=16, maximum=256, step=16, label='Latent tile height', value=96, elem_id=uid('latent-tile-height'))
|
||||
|
||||
with gr.Row(variant='compact'):
|
||||
overlap = gr.Slider(minimum=0, maximum=256, step=4, label='Latent window overlap', value=64, elem_id=uid('latent-tile-overlap-2'))
|
||||
batch_size = gr.Slider(minimum=1, maximum=8, step=1, label='Latent window batch size', value=4, elem_id=uid('latent-tile-batch-size-2'))
|
||||
overlap = gr.Slider(minimum=0, maximum=256, step=4, label='Latent window overlap', value=64, elem_id=uid('latent-tile-overlap'))
|
||||
batch_size = gr.Slider(minimum=1, maximum=8, step=1, label='Latent window batch size', value=4, elem_id=uid('latent-tile-batch-size'))
|
||||
|
||||
with gr.Row(variant='compact', visible=True) as tab_size:
|
||||
c1 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c1', value=3, elem_id=f'c1-{tab}')
|
||||
c2 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c2', value=1, elem_id=f'c2-{tab}')
|
||||
c3 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c3', value=1, visible=False, elem_id=f'c3-{tab}') #XXX:this parameter is useless in current version
|
||||
|
||||
c3 = gr.Slider(minimum=0.5, maximum=3, step=0.1, label='c3', value=1, elem_id=f'c3-{tab}')
|
||||
with gr.Row(variant='compact') as tab_upscale:
|
||||
# upscaler_name = gr.Dropdown(label='Upscaler', choices=[x.name for x in shared.sd_upscalers], value='None', elem_id=uid('upscaler-index'))
|
||||
scale_factor = gr.Slider(minimum=1.0, maximum=8.0, step=1, label='Scale_Factor', value=2.0, elem_id=uid('upscaler-factor-2'))
|
||||
# scale_factor = gr.Slider(minimum=1.0, maximum=8.0, step=1, label='Overwrite Scale Factor', value=2.0,value=is_img2img, elem_id=uid('upscaler-factor'))
|
||||
|
||||
scale_factor = gr.Slider(minimum=1.0, maximum=8.0, step=1, label='Scale Factor', value=2.0, elem_id=uid('upscaler-factor'))
|
||||
|
||||
|
||||
with gr.Accordion('Noise Inversion', open=True, visible=is_img2img) as tab_noise_inv:
|
||||
with gr.Row(variant='compact'):
|
||||
noise_inverse = gr.Checkbox(label='Enable Noise Inversion', value=False, elem_id=uid('noise-inverse-2'))
|
||||
noise_inverse_steps = gr.Slider(minimum=1, maximum=200, step=1, label='Inversion steps', value=10, elem_id=uid('noise-inverse-steps-2'))
|
||||
noise_inverse = gr.Checkbox(label='Enable Noise Inversion', value=False, elem_id=uid('noise-inverse'))
|
||||
noise_inverse_steps = gr.Slider(minimum=1, maximum=200, step=1, label='Inversion steps', value=10, elem_id=uid('noise-inverse-steps'))
|
||||
gr.HTML('<p>Please test on small images before actual upscale. Default params require denoise <= 0.6</p>')
|
||||
with gr.Row(variant='compact'):
|
||||
noise_inverse_retouch = gr.Slider(minimum=1, maximum=100, step=0.1, label='Retouch', value=1, elem_id=uid('noise-inverse-retouch-2'))
|
||||
noise_inverse_renoise_strength = gr.Slider(minimum=0, maximum=2, step=0.01, label='Renoise strength', value=1, elem_id=uid('noise-inverse-renoise-strength-2'))
|
||||
noise_inverse_renoise_kernel = gr.Slider(minimum=2, maximum=512, step=1, label='Renoise kernel size', value=64, elem_id=uid('noise-inverse-renoise-kernel-2'))
|
||||
noise_inverse_retouch = gr.Slider(minimum=1, maximum=100, step=0.1, label='Retouch', value=1, elem_id=uid('noise-inverse-retouch'))
|
||||
noise_inverse_renoise_strength = gr.Slider(minimum=0, maximum=2, step=0.01, label='Renoise strength', value=1, elem_id=uid('noise-inverse-renoise-strength'))
|
||||
noise_inverse_renoise_kernel = gr.Slider(minimum=2, maximum=512, step=1, label='Renoise kernel size', value=64, elem_id=uid('noise-inverse-renoise-kernel'))
|
||||
|
||||
# The control includes txt2img and img2img, we use t2i and i2i to distinguish them
|
||||
|
||||
|
|
@ -101,7 +92,7 @@ class Script(scripts.Script):
|
|||
noise_inverse, noise_inverse_steps, noise_inverse_retouch, noise_inverse_renoise_strength, noise_inverse_renoise_kernel,
|
||||
control_tensor_cpu,
|
||||
random_jitter,
|
||||
c1,c2,c3
|
||||
c1,c2,c3,gaussian_filter
|
||||
]
|
||||
|
||||
|
||||
|
|
@ -113,7 +104,7 @@ class Script(scripts.Script):
|
|||
noise_inverse: bool, noise_inverse_steps: int, noise_inverse_retouch: float, noise_inverse_renoise_strength: float, noise_inverse_renoise_kernel: int,
|
||||
control_tensor_cpu: bool,
|
||||
random_jitter:bool,
|
||||
c1,c2,c3
|
||||
c1,c2,c3,gaussian_filter
|
||||
):
|
||||
|
||||
# unhijack & unhook, in case it broke at last time
|
||||
|
|
@ -129,6 +120,7 @@ class Script(scripts.Script):
|
|||
p.width_original_md = p.width
|
||||
p.height_original_md = p.height
|
||||
p.current_scale_num = 1
|
||||
p.gaussian_filter = gaussian_filter
|
||||
p.scale_factor = int(scale_factor)
|
||||
|
||||
is_img2img = hasattr(p, "init_images") and len(p.init_images) > 0
|
||||
|
|
@ -136,16 +128,17 @@ class Script(scripts.Script):
|
|||
init_img = p.init_images[0]
|
||||
init_img = images.flatten(init_img, opts.img2img_background_color)
|
||||
image = init_img
|
||||
if keep_input_size: #若 scale factor为1则为真
|
||||
p.scale_factor = 1
|
||||
if keep_input_size:
|
||||
p.width = image.width
|
||||
p.height = image.height
|
||||
p.width_original_md = p.width
|
||||
p.height_original_md = p.height
|
||||
else: #XXX:To adapt to noise inversion, we do not multiply the scale factor here
|
||||
p.width = p.width_original_md
|
||||
p.height = p.height_original_md
|
||||
else: # txt2img
|
||||
p.width = p.width*(p.scale_factor)
|
||||
p.height = p.height*(p.scale_factor)
|
||||
p.width = p.width_original_md
|
||||
p.height = p.height_original_md
|
||||
|
||||
if 'png info':
|
||||
info = {}
|
||||
|
|
@ -227,31 +220,23 @@ class Script(scripts.Script):
|
|||
''' ↓↓↓ inner API hijack ↓↓↓ '''
|
||||
@torch.no_grad()
|
||||
def sample_hijack(self, conditioning, unconditional_conditioning,seeds, subseeds, subseed_strength, prompts,p,image_ori,window_size, overlap, tile_batch_size,random_jitter,c1,c2,c3):
|
||||
|
||||
if self.delegate==None:
|
||||
p.denoising_strength=1
|
||||
# p.sampler = Script.create_sampler_original_md(p.sampler_name, p.sd_model)
|
||||
p.sampler = sd_samplers.create_sampler(p.sampler_name, p.sd_model) #NOTE:Wrong but very useful. If corrected, please replace with the content from the previous line
|
||||
# 3. Encode input prompts
|
||||
shared.state.sampling_step = 0
|
||||
noise = p.rng.next()
|
||||
|
||||
if hasattr(p,'initial_noise_multiplier'):
|
||||
if p.initial_noise_multiplier != 1.0:
|
||||
p.extra_generation_params["Noise multiplier"] = p.initial_noise_multiplier
|
||||
noise *= p.initial_noise_multiplier
|
||||
|
||||
################################################## Phase Initialization ######################################################
|
||||
|
||||
if not image_ori:
|
||||
latents = p.rng.next() #Same with line 233. Replaced with the following lines
|
||||
# latents = p.sampler.sample(p, x, conditioning, unconditional_conditioning, image_conditioning=p.txt2img_image_conditioning(x))
|
||||
# del x
|
||||
# p.denoising_strength=1
|
||||
# p.sampler = sd_samplers.create_sampler(p.sampler_name, p.sd_model)
|
||||
|
||||
# p.sampler = sd_samplers.create_sampler(p.sampler_name, p.sd_model) #NOTE:Wrong but very useful. If corrected, please replace with the content with the following lines
|
||||
# latents = p.rng.next()
|
||||
p.denoising_strength = 1
|
||||
p.sampler = Script.create_sampler_original_md(p.sampler_name, p.sd_model)
|
||||
x = p.rng.next()
|
||||
latents = p.sampler.sample(p, x, conditioning, unconditional_conditioning, image_conditioning=p.txt2img_image_conditioning(x))
|
||||
del x
|
||||
p.sampler = sd_samplers.create_sampler(p.sampler_name, p.sd_model)
|
||||
starting_scale = 2
|
||||
else: # img2img
|
||||
print("### Encoding Real Image ###")
|
||||
latents = p.init_latent
|
||||
starting_scale = 1
|
||||
|
||||
|
||||
anchor_mean = latents.mean()
|
||||
|
|
@ -260,7 +245,6 @@ class Script(scripts.Script):
|
|||
devices.torch_gc()
|
||||
|
||||
####################################################### Phase Upscaling #####################################################
|
||||
starting_scale = 1
|
||||
p.cosine_scale_1 = c1 # 3
|
||||
p.cosine_scale_2 = c2 # 1
|
||||
p.cosine_scale_3 = c3 # 1
|
||||
|
|
@ -424,8 +408,6 @@ class Script(scripts.Script):
|
|||
org_random_tensors = torch.where(background_noise_count > 0, background_noise, org_random_tensors)
|
||||
org_random_tensors = torch.where(foreground_noise_count > 0, foreground_noise, org_random_tensors)
|
||||
return org_random_tensors
|
||||
# p.sd_model.sd_model_hash改为p.sd_model_hash
|
||||
''' ↓↓↓ helper methods ↓↓↓ '''
|
||||
|
||||
''' ↓↓↓ helper methods ↓↓↓ '''
|
||||
|
||||
|
|
|
|||
|
|
@ -108,10 +108,6 @@ class DemoFusion(AbstractDiffusion):
|
|||
cols=1
|
||||
dx = (w_l - tile_w) / (cols - 1) if cols > 1 else 0
|
||||
dy = (h_l - tile_h) / (rows - 1) if rows > 1 else 0
|
||||
if self.p.random_jitter:
|
||||
self.jitter_range = max((min(self.w, self.h)-self.stride)//4,0)
|
||||
else:
|
||||
self.jitter_range=0
|
||||
bbox_list: List[BBox] = []
|
||||
for row in range(rows):
|
||||
for col in range(cols):
|
||||
|
|
@ -232,8 +228,10 @@ class DemoFusion(AbstractDiffusion):
|
|||
w,h = bbox
|
||||
|
||||
######
|
||||
|
||||
x_global_i = self.sampler.model_wrap_cfg.forward_ori(self.x_in_tmp[:,:,h::self.p.current_scale_num,w::self.p.current_scale_num],sigma, **kwarg) # x_in_tmp could be changed to latents_gaussian
|
||||
if self.gaussian_filter:
|
||||
x_global_i = self.sampler.model_wrap_cfg.forward_ori(self.latents_gaussian[:,:,h::self.p.current_scale_num,w::self.p.current_scale_num],sigma, **kwarg)
|
||||
else:
|
||||
x_global_i = self.sampler.model_wrap_cfg.forward_ori(self.x_in_tmp[:,:,h::self.p.current_scale_num,w::self.p.current_scale_num],sigma, **kwarg) # x_in_tmp could be changed to latents_gaussian
|
||||
x_global[:,:,h::self.p.current_scale_num,w::self.p.current_scale_num] += x_global_i
|
||||
|
||||
######
|
||||
|
|
|
|||
Loading…
Reference in New Issue