1580158
/
SD-CN-Animation
mirror of https://github.com/volotat/SD-CN-Animation.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Note

pull/87/head
Alexey Borsky 2023-05-08 01:19:50 +03:00
parent f33a508d3f
commit c987f645d6
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
readme.md
View File

@ -1,10 +1,12 @@
# SD-CN-Animation # SD-CN-Animation
This project allows you to automate video stylization task using StableDiffusion and ControlNet. It also allows you to generate completely new videos from text at any resolution and length in contrast to other current text2video methods using any Stable Diffusion model as a backbone, including custom ones. It uses '[RAFT](https://github.com/princeton-vl/RAFT)' optical flow estimation algorithm to keep the animation stable and create an occlusion mask that is used to generate the next frame. In text to video mode it relies on 'FloweR' method (work in progress) that predicts optical flow from the previous frames. This project allows you to automate video stylization task using StableDiffusion and ControlNet. It also allows you to generate completely new videos from text at any resolution and length in contrast to other current text2video methods using any Stable Diffusion model as a backbone, including custom ones. It uses '[RAFT](https://github.com/princeton-vl/RAFT)' optical flow estimation algorithm to keep the animation stable and create an occlusion mask that is used to generate the next frame. In text to video mode it relies on 'FloweR' method (work in progress) that predicts optical flow from the previous frames.
![sd-cn-animation ui preview](examples/ui_preview.png) ![sd-cn-animation ui preview](examples/ui_preview.png)
sd-cn-animation ui preview sd-cn-animation ui preview
Note: In vid2vid mode do not forget to activate any ControlNet model to achieve better results. Without it the resulting video might be quite choppy. I personally prefer to use 'hed' model with 0.65 control strength.
### Video to Video Examples: ### Video to Video Examples:
</table> </table>
<table class="center"> <table class="center">