🦀 I curious that can we do Stable Diffusion
things with Rust
? Turn out there’s a way to do just that via FFI
which I never think I will get a chance to use it so this worth noted in my book and re-post here 👉 https://book.gist.rs/rust/r2/hello-diffusers-stable-diffusion.html
Prequisites
- Source code: diffusers-rs
HUGGINGFACE_TOKEN
: Get it from here.- weights : 1.5 or 2.1
Manual Setup
For huggingface
model v1.5
, This will manual convert the weights so you will need installed Python
, numpy
, PyTorch
.
git clone https://github.com/LaurentMazare/diffusers-rs
cd diffusers-rs
. ./scripts/download_weights.sh REPLACE_HERE_WITH_YOUR_HUGGINGFACE_TOKEN
Lazy Setup
For huggingface
model v1.5
mkdir -p data
wget https://huggingface.co/lmz/rust-stable-diffusion-v1-5/resolve/main/weights/bpe_simple_vocab_16e6.txt -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v1-5/resolve/main/weights/pytorch_model.ot -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v1-5/resolve/main/weights/unet-inpaint.ot -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v1-5/resolve/main/weights/unet.ot -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v1-5/resolve/main/weights/vae.ot -P data/
For huggingface
model v2.1
mkdir -p data
wget https://huggingface.co/lmz/rust-stable-diffusion-v2-1/resolve/main/weights/bpe_simple_vocab_16e6.txt -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v2-1/resolve/main/weights/clip_v2.1.ot -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v2-1/resolve/main/weights/unet_v2.1.ot -P data/
wget https://huggingface.co/lmz/rust-stable-diffusion-v2-1/resolve/main/weights/vae_v2.1.ot -P data/
Run
For huggingface
model v1.5
(has flag --sd-version v1-5
)
cargo run --example stable-diffusion --features clap -- --prompt "A rusty cat robot holding a fire torch." --sd-version v1-5
For huggingface
model v2.1
cargo run --example stable-diffusion --features clap -- --prompt "A rusty cat robot holding a fire torch."
Take Away
It took almost 14 minutes for 1.5
and 49 minutes for 2.1
to generate 1 pics for my old mac.
💁♂️ If you like my work please consider support me so I can have better machine (mps) to play with = more content to blog.