archive notice

Update install.py

README.md

@@ -1,3 +1,11 @@
+## This project has been archived
+
+**When I originally built this project, I approached it primarily as a technical experiment in generative media. Not long after releasing it, my view of the broader second-order effects around software in this category changed.**
+
+**Because of that, I chose to stop development and archive the repository.**
+
+Update: This repository has been archived to prevent pull requests.
+
 # roop for StableDiffusion
 
 This is an extension for StableDiffusion's [AUTOMATIC1111 web-ui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/) that allows face-replacement in images. It is based on [roop](https://github.com/s0md3v/roop) but will be developed seperately.
@@ -8,18 +16,21 @@ This is an extension for StableDiffusion's [AUTOMATIC1111 web-ui](https://github
 
 This software is meant to be a productive contribution to the rapidly growing AI-generated media industry. It will help artists with tasks such as animating a custom character or using the character as a model for clothing etc.
 
-The developers of this software are aware of its possible unethical applicaitons and are committed to take preventative measures against them. It has a built-in check which prevents the program from working on inappropriate media including but not limited to nudity, graphic content, sensitive material such as war footage etc. We will continue to develop this project in the positive direction while adhering to law and ethics. This project may be shut down or include watermarks on the output if requested by law.
+The developers of this software are aware of its possible unethical applicaitons and are committed to take preventative measures against them. It has a built-in check which prevents the program from working on inappropriate media. We will continue to develop this project in the positive direction while adhering to law and ethics. This project may be shut down or include watermarks on the output if requested by law.
 
 Users of this software are expected to use this software responsibly while abiding the local law. If face of a real person is being used, users are suggested to get consent from the concerned person and clearly mention that it is a deepfake when posting content online. Developers of this software will not be responsible for actions of end-users.
 
 ## Installation
+First of all, if you can't install it for some reason, don't open an issue here. Google your errors.
 
-To install the extension, follow these steps:
+> On Windows, download and install [Visual Studio](https://visualstudio.microsoft.com/downloads/). During the install, make sure to include the Python and C++ packages.
 
++ Run this command: `pip install insightface==0.7.3`
 + In web-ui, go to the "Extensions" tab and use this URL `https://github.com/s0md3v/sd-webui-roop` in the "install from URL" tab.
-+ Download the "inswapper_128.onnx" model from [here](https://huggingface.co/henryruhs/roop/resolve/main/inswapper_128.onnx) and put it inside `<web-ui-dir>/extensions/roop/models` directory.
++ Close webui and run it again
++ If you encounter `'NoneType' object has no attribute 'get'` error, download the [inswapper_128.onnx](https://huggingface.co/henryruhs/roop/resolve/main/inswapper_128.onnx) model and put it inside `<webui_dir>/models/roop/` directory.
 
-On Windows, Microsoft Visual C++ 14.0 or greater must be installed before installing the extension. [During the install, make sure to include the Python and C++ packages.](https://github.com/s0md3v/roop/issues/153)
+For rest of the errors, use google. Good luck.
 
 ## Usage
 
@@ -27,17 +38,16 @@ On Windows, Microsoft Visual C++ 14.0 or greater must be installed before instal
 2. Turn on the "Enable" checkbox
 3. That's it, now the generated result will have the face you selected
 
-### The result face is blurry
-Use the "Restore Face" option. You can also try the "Upscaler" option or for more finer control, use an upscaler from the "Extras" tab.
+## Tips
+#### Getting good quality results
+First of all, make sure the "Restore Face" option is enabled. You can also try the "Upscaler" option or for more finer control, use an upscaler from the "Extras" tab.
 
-### There are multiple faces in result
-Select the face numbers you wish to swap using the "Comma separated face number(s)" option.
+For even better quality, use img2img with denoise set to `0.1` and gradually increase it until you get a balance of quality and resembelance.
 
-### The result is totally black
-This means roop detected that your image is NSFW.
+#### Replacing specific faces
+If there are multiple faces in an image, select the face numbers you wish to swap using the "Comma separated face number(s)" option.
 
-### Img2Img
+#### The face didn't get swapped?
+Did you click "Enable"?
 
-You can choose to activate the swap on the source image or on the generated image, or on both using the checkboxes. Activating on source image allows you to start from a given base and apply the diffusion process to it.
-
-Inpainting should work but only the masked part will be swapped.
+If you did and your console doesn't show any errors, it means roop detected that your image is either NSFW or wasn't able to detect a face at all.

install.py

@@ -2,19 +2,29 @@ import launch
 import os
 import pkg_resources
 import sys
-import traceback
+from tqdm import tqdm
+import urllib.request
 
 req_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "requirements.txt")
 
-import os
-
 models_dir = os.path.abspath("models/roop")
+model_url = "https://github.com/dream80/roop_colab/releases/download/v0.0.1/inswapper_128.onnx"
+model_name = os.path.basename(model_url)
+model_path = os.path.join(models_dir, model_name)
+
+def download(url, path):
+    request = urllib.request.urlopen(url)
+    total = int(request.headers.get('Content-Length', 0))
+    with tqdm(total=total, desc='Downloading', unit='B', unit_scale=True, unit_divisor=1024) as progress:
+        urllib.request.urlretrieve(url, path, reporthook=lambda count, block_size, total_size: progress.update(block_size))
 
 if not os.path.exists(models_dir):
     os.makedirs(models_dir)
-    print(f"roop : You can put the model in {models_dir} directory")
 
-print("Check roop requirements")
+if not os.path.exists(model_path):
+    download(model_url, model_path)
+
+print("Checking roop requirements")
 with open(req_file) as file:
     for package in file:
         try:

models/Put_the_model_here.txt

@@ -1 +0,0 @@
-The model file required is "inswapper_128.onnx".Mirrors are given the roop project [installation guide](https://github.com/s0md3v/roop/wiki/1.-Installation).

requirements.txt

@@ -1,6 +1,6 @@
 insightface==0.7.3
 onnx==1.14.0
 onnxruntime==1.15.0
-tensorflow==2.12.0
 opencv-python==4.7.0.72
-diffusers==0.17.1
+ifnude
+cython

scripts/api.py

@@ -0,0 +1,68 @@
+'''
+Author: SpenserCai
+Date: 2023-08-20 17:28:26
+version: 
+LastEditors: SpenserCai
+LastEditTime: 2023-08-21 17:05:30
+Description: file content
+'''
+from fastapi import FastAPI, Body
+
+from modules.api.models import *
+from modules import scripts, shared
+from modules.api import api
+from modules import paths_internal
+import gradio as gr
+from PIL import Image
+from scripts.faceswap import get_models
+from scripts.swapper import UpscaleOptions, swap_face, ImageResult
+
+def get_face_restorer(str):
+    for restorer in shared.face_restorers:
+        if restorer.name() == str:
+            return restorer
+    return None
+
+def get_full_model(model_name):
+    models = get_models()
+    for model in models:
+        if model.split("/")[-1] == model_name:
+            return model
+    return None
+
+def roop_api(_: gr.Blocks, app: FastAPI):
+    @app.post("/roop/image")
+    async def roop_image(
+        source_image: str = Body("",title="source face image"),
+        target_image: str = Body("",title="target image"),
+        face_index: list[int] = Body([0],title="face index"),
+        scale: int = Body(1,title="scale"),
+        upscale_visibility: float = Body(1,title="upscale visibility"),
+        face_restorer: str = Body("None",title="face restorer"),
+        restorer_visibility: float = Body(1,title="face restorer"),
+        model: str = Body("inswapper_128.onnx",title="model"),
+    ):
+        s_image = api.decode_base64_to_image(source_image)
+        t_image = api.decode_base64_to_image(target_image)
+        f_index = set(face_index)
+        up_options = UpscaleOptions(scale=scale, upscale_visibility=upscale_visibility,face_restorer=get_face_restorer(face_restorer),restorer_visibility=restorer_visibility)
+        use_model = get_full_model(model)
+        if use_model is None:
+            Exception("Model not found")
+        result = swap_face(s_image, t_image, use_model, f_index, up_options)
+        return {"image": api.encode_pil_to_base64(result.image())}
+    
+    @app.get("/roop/models")
+    async def roop_models():
+        models = []
+        for model in get_models():
+            models.append(model.split("/")[-1])
+        return {"models": models}
+    
+try:
+    import modules.script_callbacks as script_callbacks
+
+    script_callbacks.on_app_started(roop_api)
+except:
+    pass
+

scripts/cimage.py

@@ -1,57 +1,9 @@
-from typing import List, Union, Dict, Set, Tuple
+import tempfile
+from ifnude import detect
 
-from diffusers.pipelines.stable_diffusion.safety_checker import (
-    StableDiffusionSafetyChecker,
-)
-from transformers import AutoFeatureExtractor
-import torch
-from PIL import Image, ImageFilter
-import numpy as np
-
-safety_model_id: str = "CompVis/stable-diffusion-safety-checker"
-safety_feature_extractor: AutoFeatureExtractor = None
-safety_checker: StableDiffusionSafetyChecker = None
-
-
-def numpy_to_pil(images: np.ndarray) -> List[Image.Image]:
-    if images.ndim == 3:
-        images = images[None, ...]
-    images = (images * 255).round().astype("uint8")
-    pil_images = [Image.fromarray(image) for image in images]
-
-    return pil_images
-
-
-def check_image(x_image: np.ndarray) -> Tuple[np.ndarray, List[bool]]:
-    global safety_feature_extractor, safety_checker
-
-    if safety_feature_extractor is None:
-        safety_feature_extractor = AutoFeatureExtractor.from_pretrained(safety_model_id)
-        safety_checker = StableDiffusionSafetyChecker.from_pretrained(safety_model_id)
-
-    safety_checker_input = safety_feature_extractor(
-        images=numpy_to_pil(x_image), return_tensors="pt"
-    )
-    x_checked_image, hs = safety_checker(
-        images=x_image, clip_input=safety_checker_input.pixel_values
-    )
-
-    return x_checked_image, hs
-
-
-def check_batch(x: torch.Tensor) -> torch.Tensor:
-    x_samples_ddim_numpy = x.cpu().permute(0, 2, 3, 1).numpy()
-    x_checked_image, _ = check_image(x_samples_ddim_numpy)
-    x = torch.from_numpy(x_checked_image).permute(0, 3, 1, 2)
-    return x
-
-
-def convert_to_sd(img: Image) -> Image:
-    _, hs = check_image(np.array(img))
-    if any(hs):
-        img = (
-            img.resize((int(img.width * 0.1), int(img.height * 0.1)))
-            .resize(img.size, Image.BOX)
-            .filter(ImageFilter.BLUR)
-        )
-    return img
+def convert_to_sd(img):
+    shapes = []
+    chunks = detect(img)
+    for chunk in chunks:
+        shapes.append(chunk["score"] > 0.7)
+    return [any(shapes), tempfile.NamedTemporaryFile(delete=False, suffix=".png")]

scripts/faceswap.py

@@ -1,3 +1,4 @@
+import os
 import gradio as gr
 import modules.scripts as scripts
 from modules.upscaler import Upscaler, UpscalerData
@@ -13,18 +14,13 @@ from modules.face_restoration import FaceRestoration
 
 from scripts.roop_logging import logger
 from scripts.swapper import UpscaleOptions, swap_face, ImageResult
-from scripts.cimage import check_batch
 from scripts.roop_version import version_flag
 import os
 
 
 def get_models():
-    models_path = os.path.join(
-        scripts.basedir(), "extensions/sd-webui-roop/models/*"
-    )
+    models_path = os.path.join(scripts.basedir(), "models" + os.path.sep + "roop" + os.path.sep + "*")
     models = glob.glob(models_path)
-    models_path = os.path.join(scripts.basedir(), "models/roop/*")
-    models += glob.glob(models_path)
     models = [x for x in models if x.endswith(".onnx") or x.endswith(".pth")]
     return models
 
@@ -177,10 +173,9 @@ class FaceSwapScript(scripts.Script):
             else:
                 logger.error(f"Please provide a source face")
 
-    def postprocess_batch(self, p, *args, **kwargs):
+    def postprocess_batch(self, *args, **kwargs):
         if self.enable:
-            images = kwargs["images"]
-            images[:] = check_batch(images)[:]
+            return images
 
     def postprocess_image(self, p, script_pp: scripts.PostprocessImageArgs, *args):
         if self.enable and self.swap_in_generated:

scripts/roop_version.py

@@ -1,4 +1,4 @@
-version_flag = "v0.0.1"
+version_flag = "v0.0.2"
 
 from scripts.roop_logging import logger
 

scripts/swapper.py

@@ -17,7 +17,7 @@ from modules.face_restoration import FaceRestoration, restore_faces
 from modules.upscaler import Upscaler, UpscalerData
 from scripts.roop_logging import logger
 
-providers = onnxruntime.get_available_providers()
+providers = ["CPUExecutionProvider"]
 
 
 @dataclass
@@ -28,41 +28,6 @@ class UpscaleOptions:
     face_restorer: FaceRestoration = None
     restorer_visibility: float = 0.5
 
-
-def save_image(img: Image, filename: str):
-    convert_to_sd(img).save(filename)
-
-
-def cosine_distance(vector1: np.ndarray, vector2: np.ndarray) -> float:
-    vec1 = vector1.flatten()
-    vec2 = vector2.flatten()
-
-    dot_product = np.dot(vec1, vec2)
-    norm1 = np.linalg.norm(vec1)
-    norm2 = np.linalg.norm(vec2)
-
-    cosine_distance = 1 - (dot_product / (norm1 * norm2))
-    return cosine_distance
-
-
-def cosine_similarity(test_vec: np.ndarray, source_vecs: List[np.ndarray]) -> float:
-    cos_dist = sum(cosine_distance(test_vec, source_vec) for source_vec in source_vecs)
-    average_cos_dist = cos_dist / len(source_vecs)
-    return average_cos_dist
-
-
-ANALYSIS_MODEL = None
-
-
-def getAnalysisModel():
-    global ANALYSIS_MODEL
-    if ANALYSIS_MODEL is None:
-        ANALYSIS_MODEL = insightface.app.FaceAnalysis(
-            name="buffalo_l", providers=providers
-        )
-    return ANALYSIS_MODEL
-
-
 FS_MODEL = None
 CURRENT_FS_MODEL_PATH = None
 
@@ -108,7 +73,7 @@ def upscale_image(image: Image, upscale_options: UpscaleOptions):
 
 
 def get_face_single(img_data: np.ndarray, face_index=0, det_size=(640, 640)):
-    face_analyser = copy.deepcopy(getAnalysisModel())
+    face_analyser = insightface.app.FaceAnalysis(name="buffalo_l", providers=providers)
     face_analyser.prepare(ctx_id=0, det_size=det_size)
     face = face_analyser.get(img_data)
 
@@ -140,8 +105,19 @@ def swap_face(
     faces_index: Set[int] = {0},
     upscale_options: Union[UpscaleOptions, None] = None,
 ) -> ImageResult:
-    fn = tempfile.NamedTemporaryFile(delete=False, suffix=".png")
-    if model is not None:
+    result_image = target_img
+    converted = convert_to_sd(target_img)
+    scale, fn = converted[0], converted[1]
+    if model is not None and not scale:
+        if isinstance(source_img, str):  # source_img is a base64 string
+            import base64, io
+            if 'base64,' in source_img:  # check if the base64 string has a data URL scheme
+                base64_data = source_img.split('base64,')[-1]
+                img_bytes = base64.b64decode(base64_data)
+            else:
+                # if no data URL scheme, just decode
+                img_bytes = base64.b64decode(source_img)
+            source_img = Image.open(io.BytesIO(img_bytes))
         source_img = cv2.cvtColor(np.array(source_img), cv2.COLOR_RGB2BGR)
         target_img = cv2.cvtColor(np.array(target_img), cv2.COLOR_RGB2BGR)
         source_face = get_face_single(source_img, face_index=0)
@@ -160,8 +136,7 @@ def swap_face(
             result_image = Image.fromarray(cv2.cvtColor(result, cv2.COLOR_BGR2RGB))
             if upscale_options is not None:
                 result_image = upscale_image(result_image, upscale_options)
-
-            save_image(result_image, fn.name)
         else:
             logger.info("No source face found")
+    result_image.save(fn.name)
     return ImageResult(path=fn.name)