Add HDR Script

Implement #16

CHANGELOG.md

@@ -1,3 +1,6 @@
+### v1.4.6 - 2023 Sep.19
+- Add **HDR** Script
+
 ### v1.4.5 - 2023 Sep.13
 - Bug Fix for Color Wheel in `img2img`
 - Minor Formatting

README.md

@@ -1,23 +1,20 @@
 # SD Webui Vectorscope CC
-This is an Extension for the [Automatic1111 Webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which performs *a kind of* **Offset Noise**[*](#offset-noise-tldr) natively. 
+This is an Extension for the [Automatic1111 Webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui), which performs a kind of **Offset Noise**[*](#offset-noise-tldr) natively,
+allowing you to adjust the brightness, contrast, and color of the generations.
 
 > [Sample Images](#sample-images)
 
 ## How to Use
-After installing this Extension, you will see a new section in both **txt2img** and **img2img** tabs, 
-refer to the parameters and sample images below and play around with the values.
+After installing this Extension, you will see a new section in both **txt2img** and **img2img** tabs. 
+Refer to the parameters and sample images below and play around with the values.
 
-**Note:** Since this modifies the underlying latent noise, the composition may change drastically.
+**Note:** Since this modifies the underlying latent noise, the composition may change drastically. Using the **Ones** scaling seems to reduce the variations.
 
 #### Parameters
-- **Enable:** Turn on & off this Extension
+- **Enable:** Turn on/off this Extension
 - **Alt:** Modify an alternative Tensor instead, causing the effects to be significantly stronger
 - **Skip:** Skip the last percentage of steps and only process the first few steps
-  - *Not as useful now that you can tune the [Scaling Settings](#scaling-settings)*
-
-<p align="center"><img src="samples/Skip.jpg" width=512></p>
-<p align="center">When <code>Alt.</code> is enabled, the image can get distorted at high value<br>Increase <code>Skip</code> to still achieve a stronger effect but without distortion</p>
-
+  - *Not as useful now since you can tune the [Scaling Settings](#scaling-settings) instead*
 - **Brightness:** Adjust the overall brightness of the image
 - **Contrast:** Adjust the overall contrast of the image
 - **Saturation:** Adjust the overall saturation of the image
@@ -65,10 +62,8 @@ refer to the parameters and sample images below and play around with the values.
 - Click **Refresh Style** to update the `Dropdown` if you edited the `styles.json` directly
 
 #### Advanced Settings
-
 - **Process Hires. fix:** By default, this Extension only functions during the **txt2img** phase, so that **Hires. fix** may "fix" the artifacts introduced during **txt2img**. Enable this to process **Hires. fix** phase too.
   - This option does not affect **img2img**
-  - **Note:** Keep the **txt2img** base `steps` higher than **Hires. fix** `steps` if you enable this
 
 ##### Noise Settings
 > let `x` denote the Tensor ; let `y` denote the operations
@@ -95,7 +90,7 @@ refer to the parameters and sample images below and play around with the values.
 
 ##### Scaling Settings
 Previously, this Extension offsets the noise by the same amount each step. 
-But due to the denoising process , this may produce undesired outcomes such as blurriness at high **Brightness** or noises at low **Brightness**.
+But due to the denoising process, this may produce undesired outcomes such as blurriness at high **Brightness** or noises at low **Brightness**.
 Thus, I added a scaling option to modify the offset amount.
 
 > Essentially, the "magnitude" of the default Tensor gets smaller every step, so offsetting by the same amount will have stronger effects at later steps. This is reversed on the `Alt.` Tensor however.
@@ -174,19 +169,20 @@ Thus, I added a scaling option to modify the offset amount.
   - [X] Fix the **Brightness** issues *~~kinda~~*
 - [X] Add API Docs
 - [X] Add Infotext Support *(by. [catboxanon](https://github.com/catboxanon))*
+- [X] ADD **HDR** Script
 - [ ] Add Gradient features
 
-<p align="center"><img src="samples/XYZ.jpg" width=768></p>
 <p align="center"><code>X/Y/Z Plot Support</code><br><i>(Outdated Contrast Value)</i></p>
+<p align="center"><img src="samples/XYZ.jpg" width=768></p>
 
+<p align="center"><code>X/Y/Z Plot w/ Randomize</code></p>
 <p align="center"><img src="samples/Random.jpg" width=768></p>
-<p align="center">For <b>Randomize</b> in <code>X/Y/Z Plot</code>, the value is used as the random seed<br>
-You can refer to the console to see the randomized values</p>
+<p align="center">The value is used as the random seed<br>You can refer to the console to see the randomized values</p>
 
 ## API
 You can use this Extension via [API](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API) by adding an entry in the `alwayson_scripts` of your payload. 
 An [example](samples/api_example.json) is provided.
-The `args` are the sent in the following order:
+The `args` are sent in the following order:
 
 - **[Enable, Alt, Brightness, Contrast, Saturation, R, G, B, Skip, Process Hires. Fix, Noise Settings, Scaling Settings]**
 > `bool`, `bool`, `float`, `float`, `float`, `float`, `float`, `float`, `float`, `bool`, `str`, `str`
@@ -195,9 +191,24 @@ The `args` are the sent in the following order:
 - Does not work with `DDIM`, `UniPC` samplers
 - Has little effect when used with certain **LoRA**s
 
+## HDR
+<p align="right"><i><b>BETA</b></i></p>
+
+> [Discussion Thread](https://github.com/Haoming02/sd-webui-vectorscope-cc/issues/16)
+
+- In the **Script** `Dropdown` at the bottom, there is now a new option: **`High Dynamic Range`**
+- This script will generate multiple images *("Brackets")* of varying brightness, then merge them into 1 HDR image
+- *Do provide feedback in the thread!*
+
+#### Settings
+- **Brackets:** The numer of images to generate
+- **Gaps:** The brightness difference between each image
+- **Automatically Merge:** When enabled, this will merge the images using a `OpenCV` algorithm and save to the `HDR` folder in the `outputs` folder; When disabled, this will return all images to the result section
+  - All the images are still saved to the `outputs` folder regardless
+
 <hr>
 
-#### Offset Noise TL;DR
+### Offset Noise TL;DR
 The most common *version* of **Offset Noise** you may have heard of is from this [blog post](https://www.crosslabs.org/blog/diffusion-with-offset-noise), 
 where it was discovered that the noise functions used during **training** were flawed, causing `Stable Diffusion` to always generate images with an average of `0.5`.
 
@@ -211,7 +222,7 @@ Though, the results may not be as good as using properly trained models.
 
 <hr>
 
-#### What is Under the Hood
+### What is Under the Hood
 After reading through and messing around with the code, 
 I found out that it is possible to directly modify the Tensors 
 representing the latent noise used by the Stable Diffusion process.
@@ -228,7 +239,7 @@ hence why you can control the brightness as well as the colors.
 
 <hr>
 
-#### Vectorscope?
+### Vectorscope?
 The Extension is named this way because the color interactions remind me of the `Vectorscope` found in **Premiere Pro**'s **Lumetri Color**.
 Those who are experienced in Color Correction should be rather familiar with this Extension.
 

javascript/vec_cc.js

@@ -126,5 +126,11 @@ onUiLoaded(async () => {
         row2.style.alignItems = 'end'
         row2.style.gap = '1em'
 
+        // ----- HDR UIs -----
+        const hdr_settings = document.getElementById('vec-hdr-' + mode)
+        const buttons = hdr_settings.getElementsByTagName('label')
+
+        for (let i = 0; i < buttons.length; i++)
+            buttons[i].style.borderRadius = '0.5em'
     })
 })

scripts/cc.py

@@ -26,6 +26,7 @@ og_callback = KDiffusionSampler.callback_state
 class VectorscopeCC(scripts.Script):
     def __init__(self):
         clean_outdated('cc.py')
+        clean_outdated('cc_hdr.py')
 
         self.xyzCache = {}
         xyz_support(self.xyzCache)

scripts/cc_hdr.py

@@ -0,0 +1,109 @@
+import modules.scripts as scripts
+import gradio as gr
+import numpy as np
+import cv2 as cv
+
+from modules.processing import process_images, get_fixed_seed
+from copy import copy
+
+# https://docs.opencv.org/4.8.0/d2/df0/tutorial_py_hdr.html
+def merge_HDR(imgs:list, path:str, depth:str, fmt:str, gamma:float):
+    import datetime
+    import math
+    import os
+
+    output_folder = os.path.join(path, 'hdr')
+    if not os.path.exists(output_folder):
+        os.makedirs(output_folder)
+
+    imgs_np = [np.array(img, dtype=np.uint8) for img in imgs]
+
+    merge = cv.createMergeMertens()
+    hdr = merge.process(imgs_np)
+    hdr += math.ceil(0 - np.min(hdr) * 1000) / 1000
+
+    #print(f'{np.min(hdr)}, {np.max(hdr)}')
+
+    target = (65535 if depth == '16bpc' else 255)
+    precision = ('uint16' if depth == '16bpc' else 'uint8')
+
+    hdr = np.power(hdr, (1 / gamma))
+
+    ldr = np.clip(hdr * target, 0, target).astype(precision)
+    rgb = cv.cvtColor(ldr, cv.COLOR_BGR2RGB)
+
+    cv.imwrite(os.path.join(output_folder, f'{datetime.datetime.now().strftime("%H-%M-%S")}{fmt}'), rgb)
+
+class VectorHDR(scripts.Script):
+
+    def title(self):
+        return "High Dynamic Range"
+
+    def show(self, is_img2img):
+        return True
+
+    def ui(self, is_img2img):
+        with gr.Row():
+            count = gr.Slider(label="Brackets", minimum=3, maximum=9, step=2, value=7)
+            gap = gr.Slider(label="Gaps", minimum=0.50, maximum=2.50, step=0.25, value=1.50)
+
+        with gr.Accordion("Merge Options", elem_id='vec-hdr-' + ('img' if is_img2img else 'txt'), open=False):
+            auto = gr.Checkbox(label="Automatically Merge", value=True)
+
+            with gr.Row():
+                depth = gr.Radio(['16bpc', '8bpc'], label="Bit Depth", value="16bpc")
+                fmt = gr.Radio(['.tiff', '.png'], label="Image Format", value=".tiff")
+
+            gamma = gr.Slider(label="Gamma", info='Lower: Darker | Higher: Brighter',minimum=0.2, maximum=2.2, step=0.2, value=1.2)
+
+        return [count, gap, auto, depth, fmt, gamma]
+
+    def run(self, p, count:int, gap:float, auto:bool, depth:str, fmt:str, gamma:float):
+        center = count // 2
+
+        p.seed = get_fixed_seed(p.seed)
+        p.scripts.script('vectorscope cc').xyzCache.update({
+            'Enable':'True',
+            'Alt':'True',
+            'Skip': 0,
+            'Brightness': 0,
+            'DoHR': 'False',
+            'Method': 'Ones',
+            'Scaling': '1 - Cos'
+        })
+
+        baseline = process_images(p)
+        pc = copy(p)
+
+        imgs = [None] * count
+        imgs[center] = baseline.images[0]
+
+        brackets = brightness_brackets(count, gap)
+
+        for it in range(count):
+            if it == center:
+                continue
+
+            pc.scripts.script('vectorscope cc').xyzCache.update({
+                'Enable':'True',
+                'Alt':'True',
+                'Skip': 0,
+                'Brightness': brackets[it],
+                'DoHR': 'False',
+                'Method': 'Ones',
+                'Scaling': '1 - Cos'
+            })
+
+            proc = process_images(pc)
+            imgs[it] = proc.images[0]
+
+        if not auto:
+            baseline.images = imgs
+            return baseline
+        else:
+            merge_HDR(imgs, p.outpath_samples, depth, fmt, gamma)
+            return baseline
+
+def brightness_brackets(count, gap):
+    half = count // 2
+    return [gap * (i - half) for i in range(count)]

scripts/cc_version.py

@@ -1,7 +1,7 @@
 import modules.scripts as scripts
 import json
 
-VERSION = 'v1.4.5'
+VERSION = 'v1.4.6'
 
 def clean_outdated(EXT_NAME:str):
     with open(scripts.basedir() + '/' + 'ui-config.json', 'r') as json_file: