Use Riffusion model to produce music in gradio. To replicate original interpolation technique, input the prompt travel extension output frames into the riffusion tab.

Go to file

artificial a9b48c6e5a Fix torch issue (by @GeekyGhost)		2023-05-29 15:08:19 -07:00
scripts	Added visualization for beat finder	2023-04-29 23:48:26 +02:00
static	Prompt travelling guide, added Riffusion tab to manually convert all images in a folder to wav files	2022-12-19 07:15:46 +00:00
.gitignore	Genesis	2022-12-16 02:24:35 +00:00
README.MD	Update README.MD	2022-12-19 07:50:07 +00:00
install.py	Fix torch issue (by @GeekyGhost)	2023-05-29 15:08:19 -07:00
license.txt	README, license	2022-12-16 02:33:19 +00:00
requirements.txt	Replace pydub with pedalboard (removes ffmpeg issue for people who don't have it), also option to preserve original WAV file alongside the mp3	2022-12-17 18:42:30 +00:00

README.MD

Riffusion extension for AUTOMATIC1111 Web UI

Installation

Make sure ffmpeg is installed and the folder with the binaries is in your PATH
Clone this repo inside your /extensions folder, or use the Install from URL functionality in the UI

Usage

Select the Riffusion Audio Generator script before generating, and use the riffusion model.

You can also convert a whole folder of images to audio in the Riffusion tab.

Prompt Travelling

If you want to prompt travel in the latent space as described by the authors, install this extension:

https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel

It will output the results of runs in the <SD>/outputs/(txt|img)2img-images/prompt_travel/ directory, and you can use the convert folder to audio functionality in the Riffusion tab to generate a single stitched-together audio file alongside the individual ones.

Here is a sample made by travelling in img2img mode from jamaican rap to deep house, techno with denoise 0.5 for 14 steps, and using the og_beat.png provided by the original authors as a base image:

Audio Sample (Jamaican Rap to Deep House, Techno)

Acknowledgements

Credit to the original Riffusion authors, Seth Forsgren and Hayk Martiros:

https://riffusion.com/about