CLIP Guided Diffusion

From @crowsonkb.

The Replicate demo runs the version from October 2022, which has none of the speedups, models or options this README measures.

Disclaimer: I’m redirecting efforts to pyglide and may be slow to address bugs here.

I also recommend looking at @crowsonkb’s v-diffusion-pytorch.

See more generations, each with its command, in the Gallery.

Install

Requires uv.

git clone https://github.com/afiaka87/clip-guided-diffusion.git
cd clip-guided-diffusion
uv sync

Run

uv run cgd -gif -freq 3 -size 256 -clip ViT-B/32+PE-Core-B-16 -n 90 -gl 0.99 -cutn 4 "A monarch butterfly on a purple flower"

That is the classic pipeline: an ImageNet model steered by CLIP, with no extra dependencies to download. It conditions the model on the ImageNet class closest to the prompt, and prints it: monarch butterfly, here. `uv run cgd -gif “A monarch…