Init
Haoming
parent
039858cd78
commit
93b87342f1
|
|
@ -0,0 +1 @@
|
|||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
113
README.md
113
README.md
|
|
@ -1,2 +1,111 @@
|
|||
# sd-webui-vectorscope-cc
|
||||
An Extension for Automatic1111 Webui that performs noise offset natively
|
||||
# SD Webui Vectorscope CC
|
||||
This is an Extension for the [Automatic1111 Webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which performs *~~some kind of~~ **Offset Noise***[*](#offset-noise-tldr) natively.
|
||||
|
||||
***Note:** This Extension is the result of my numerous trial and error[*](#what-is-actually-under-the-hood). I have no idea how and why this works.* 💀
|
||||
|
||||
## How to Use
|
||||
After installing this Extension, you will a new section in both **txt2img** and **img2img** tabs.
|
||||
Refer to the parameters and sample images below, and play around with the values.
|
||||
|
||||
**Note:** Since this modifies the underlying latent noise, the composition may change drastically.
|
||||
|
||||
#### Parameters
|
||||
- **Enable:** Turn on & off this Extension
|
||||
- **Alt:** Modify an alternative Tensor. The effects are significantly stronger when this is **on**.
|
||||
- **Brightness:** Adjust the overall brightness of the image
|
||||
- **Note:** Value too **high** causes the image to be blurry; Value too **low** causes extra noises to appear.
|
||||
- **Contrast:** Adjust the overall contrast of the image
|
||||
- **Saturation:** Adjust the overall saturation of the image
|
||||
- **Skip:** Only process the first few steps and skip the rest, to combat the issues mentioned in **Brightness**
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr align="center">
|
||||
<td>Parameter</td>
|
||||
<td>Decrease</td>
|
||||
<td>Increase</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>R</td>
|
||||
<td>Cyan</td>
|
||||
<td>Red</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>G</td>
|
||||
<td>Magenta</td>
|
||||
<td>Green</td>
|
||||
</tr>
|
||||
<tr align="center">
|
||||
<td>B</td>
|
||||
<td>Yellow</td>
|
||||
<td>Blue</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## Sample Images
|
||||
- **Checkpoint:** [UHD-23](https://civitai.com/models/22371/uhd-23)
|
||||
- **Pos. Prompt:** `(masterpiece, best quality), 1girl, solo, night, street, city, neon_lights`
|
||||
- **Neg. Prompt:** `(low quality, worst quality:1.2), `[`EasyNegative`](https://huggingface.co/datasets/gsdf/EasyNegative/tree/main)`, `[`EasyNegativeV2`](https://huggingface.co/gsdf/Counterfeit-V3.0/tree/main/embedding)
|
||||
- `Euler a`; `20 steps`; `7.5 CFG`; `Hires. fix`; `Latent (nearest)`; `16 H.steps`; `0.6 D.Str.`; `Seed:`**`3814649974`**
|
||||
|
||||
<p align="center"><b>Base</b><br>
|
||||
Extension <code>Disabled</code><br>
|
||||
<img src="samples/00.png" width=512></p>
|
||||
|
||||
<p align="center"><b>Dark</b><br>
|
||||
<b>Brightness:</b><code>-2</code>; <b>Contrast:</b><code>1</code><br>
|
||||
<img src="samples/01.png" width=512></p>
|
||||
|
||||
<p align="center"><b>Bright</b><br>
|
||||
<b>Brightness:</b><code>1</code>; <b>Contrast:</b><code>-0.5</code>; <b>Alt:</b><code>Enabled</code><br>
|
||||
<img src="samples/02.png" width=512></p>
|
||||
|
||||
<p align="center"><b>Chill</b><br>
|
||||
<b>Brightness:</b><code>-2.5</code>; <b>Contrast:</b><code>1.5</code><br>
|
||||
<b>R:</b><code>-0.5</code>; <b>B:</b><code>1</code><br>
|
||||
<img src="samples/03.png" width=512></p>
|
||||
|
||||
<p align="center"><b><s>Mexico Movie</s></b><br>
|
||||
<b>Brightness:</b><code>3</code>; <b>Contrast:</b><code>-1.5</code>; <b>Saturation:</b><code>1</code><br>
|
||||
<b>R:</b><code>1</code>; <b>G:</b><code>0.5</code>; <b>B:</b><code>-2</code><br>
|
||||
<img src="samples/04.png" width=512></p>
|
||||
|
||||
## Roadmap
|
||||
- [X] Extension Released
|
||||
- [ ] Add Support for **X/Y/Z Plot**
|
||||
- [ ] Append Parameters onto Metadata
|
||||
|
||||
<hr>
|
||||
|
||||
#### Offset Noise TL;DR
|
||||
It was [discovered](https://www.crosslabs.org/blog/diffusion-with-offset-noise) that, the noise functions used during training were flawed, causing Stable Diffusion to always generate images with an average brightness of `0.5`.
|
||||
|
||||
> **ie.** Even if you prompt for dark/night or bright/snow, the overall image still looks "grey" on average
|
||||
|
||||
> [Technical Explanations](https://youtu.be/cVxQmbf3q7Q)
|
||||
|
||||
<hr>
|
||||
|
||||
#### What is Actually Under the Hood
|
||||
After reading through and messing around with the code,
|
||||
I found out that it is possible to modify the Tensors
|
||||
representing the latent noise used by the Stable Diffusion process.
|
||||
|
||||
The dimentions of the Tensors is `(1, 4, 64, 64)`, which can be thought of like this:
|
||||
|
||||
> **1** noise image, with **4** channels, each with **64x64** values
|
||||
|
||||
So I tried to play around with the values of each channel,
|
||||
and ended up discovering these relationships between channels and colors.
|
||||
Thus, I wrote it as an Extension.
|
||||
|
||||
*~~And why does it work this way? IDK~~*
|
||||
|
||||
<hr>
|
||||
|
||||
#### Vectorscope?
|
||||
The Extension is named this way because the color relations remind me of the `Vectorscope` in **Premiere Pro**.
|
||||
Those who are experienced in Color Correction should be rather familiar with this Extension.
|
||||
|
||||
<p align="center"><img src="samples/Vectorscope.png" width=256></p>
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -0,0 +1,109 @@
|
|||
from modules.sd_samplers_kdiffusion import KDiffusionSampler
|
||||
import modules.scripts as scripts
|
||||
from modules import shared
|
||||
import gradio as gr
|
||||
import torch
|
||||
|
||||
class VectorscopeCC(scripts.Script):
|
||||
def __init__(self):
|
||||
global og_callback
|
||||
og_callback = KDiffusionSampler.callback_state
|
||||
|
||||
def title(self):
|
||||
return "Vectorscope Color Correction"
|
||||
|
||||
def show(self, is_img2img):
|
||||
return scripts.AlwaysVisible
|
||||
|
||||
def ui(self, is_img2img):
|
||||
with gr.Accordion("Vectorscope CC", open=False):
|
||||
|
||||
with gr.Row() as basics:
|
||||
enable = gr.Checkbox(label="Enable")
|
||||
latent = gr.Checkbox(label="Alt.")
|
||||
|
||||
with gr.Row() as BnC:
|
||||
bri = gr.Slider(label="Brightness", minimum=-5.0, maximum=5.0, step=0.05, value=0.0)
|
||||
con = gr.Slider(label="Contrast", minimum=-2.0, maximum=2.0, step=0.05, value=0.0)
|
||||
|
||||
with gr.Row() as SS:
|
||||
sat = gr.Slider(label="Saturation", minimum=-2.0, maximum=2.0, step=0.05, value=0.0)
|
||||
early = gr.Slider(label="Skip", minimum=0.0, maximum=1.0, step=0.05, value=0.1)
|
||||
|
||||
with gr.Row() as RGB:
|
||||
r = gr.Slider(label="R", info='Cyan | Red', minimum=-2.0, maximum=2.0, step=0.05, value=0.0)
|
||||
g = gr.Slider(label="G", info='Magenta | Green',minimum=-2.0, maximum=2.0, step=0.05, value=0.0)
|
||||
b = gr.Slider(label="B", info='Yellow | Blue',minimum=-2.0, maximum=2.0, step=0.05, value=0.0)
|
||||
|
||||
return [enable, latent, bri, con, sat, early, r, g, b]
|
||||
|
||||
def process(self, p, enable:bool, latent:bool, bri:float, con:float, sat:float, early:float, r:float, g:float, b:float):
|
||||
if not enable:
|
||||
setattr(KDiffusionSampler, "callback_state", og_callback)
|
||||
return p
|
||||
|
||||
steps = p.steps
|
||||
if not hasattr(p, 'enable_hr') and hasattr(p, 'denoising_strength') and not shared.opts.img2img_fix_steps:
|
||||
steps = int(steps * p.denoising_strength)
|
||||
|
||||
stop = int(steps * (1 - early))
|
||||
|
||||
if stop < 1:
|
||||
return p
|
||||
|
||||
bri /= steps
|
||||
con = 1.0 + con / steps
|
||||
sat = 1.0 + sat / steps
|
||||
r /= steps
|
||||
g /= steps
|
||||
b /= steps
|
||||
|
||||
mode = 'x' if latent else 'denoised'
|
||||
|
||||
# Channel 0: Dark | Bright
|
||||
# Channel 1: Purple | Green
|
||||
# Channel 2: Red | Cyan
|
||||
# Channel 2: Violet | Yellow
|
||||
|
||||
def callback_state(self, d):
|
||||
if hasattr(p, 'enable_hr') and p.enable_hr:
|
||||
if not hasattr(p, 'hr_pass'):
|
||||
p.hr_pass = 0
|
||||
|
||||
if p.hr_pass == 0:
|
||||
if d["i"] == 0:
|
||||
p.hr_pass = 1
|
||||
|
||||
elif p.hr_pass == 1:
|
||||
if d["i"] == 0:
|
||||
p.hr_pass = 2
|
||||
|
||||
if p.hr_pass == 2:
|
||||
return og_callback(self, d)
|
||||
|
||||
if d["i"] > stop:
|
||||
return og_callback(self, d)
|
||||
|
||||
BRIGHTNESS = d[mode][0, 0]
|
||||
R = d[mode][0, 2]
|
||||
G = d[mode][0, 1]
|
||||
B = d[mode][0, 3]
|
||||
|
||||
BRIGHTNESS += torch.abs(BRIGHTNESS) * bri
|
||||
BRIGHTNESS *= con
|
||||
|
||||
R -= r
|
||||
G += g
|
||||
B -= b
|
||||
|
||||
R *= sat
|
||||
G *= sat
|
||||
B *= sat
|
||||
|
||||
return og_callback(self, d)
|
||||
|
||||
setattr(KDiffusionSampler, "callback_state", callback_state)
|
||||
return p
|
||||
|
||||
def postprocess(self, p, processed, *args):
|
||||
setattr(KDiffusionSampler, "callback_state", og_callback)
|
||||
Loading…
Reference in New Issue