5.1 KiB

Raw Blame History

depthmap2mask

Made as a script for the AUTOMATIC1111/stable-diffusion-webui repository.

💥 Installation 💥

Copy the url of that repository into the extension tab :

_{just ignore the fact that this is the URL of my other repository}

OR copy that repository in your extension folder :

_{just ignore the fact that this is the name of my other repository. That one will be named depthmap2mask.}

You might need to restart the whole UI. Maybe twice.

The look

What does this extension do?

It creates masks for img2img based on a depth estimation made by MiDaS.

Explanations of the different UI elements

Contrasts cut level

This slider is purely optional. The depthmap is in levels of gray. Each pixel has a value in between 0 and 255 depending if they are black (0) or white (255). That threshold slider will cut to black every pixel below the selected value and scale from black to white what is above its value.

Or in a more human language, it will give more depth to your depthmaps while removing a lot of information.

Example before/after using the MiDaS-Large model (value around 220):

Using the MiDaS small model will give you similar if not more interesting results.

So that's more of an extra-extra option or a way to make sure that your backgrounds are untouched by using a low value (like 50).

Match input size/Net width/Net height

Match input size (On by default) will make the depth analysis at the same size as the original image. Better not to touch it unless you are having performance issues.

The sliders below will be the resolution of the analysis if Match input size is turned off.

You can also just use these functionalities to test out different results.

Misc options

Override options :

These two options simply overrides the inpainting Masked content method and mask blur. I added these because using "original" for Masked content and Mask Blur at 0 just works better. This saves you the clics needed to switch to the intpaint tab/reupload the image to that tab and select the right options.
MiDaS models :

I'll let you try what suits your needs the most.

Credits/Citation

Thanks to thygate for letting me blatantly copy-paste some of his functions for the depth analysis integration in the webui.

This repository runs with MiDaS.

@ARTICLE {Ranftl2022,
    author  = "Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun",
    title   = "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer",
    journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
    year    = "2022",
    volume  = "44",
    number  = "3"
}

@article{Ranftl2021,
	author    = {Ren\'{e} Ranftl and Alexey Bochkovskiy and Vladlen Koltun},
	title     = {Vision Transformers for Dense Prediction},
	journal   = {ICCV},
	year      = {2021},
}

Examples using different MiDaS models and denoising strength

I forgot my settings but in the end it's all pretty easy to guess what you need.

5.1 KiB Raw Blame History