What Is VAE Stable Diffusion

Are you having a difficult time generating clear images on Stable Diffusion? Learn how to get the best images through the steps below.
What Is VAE Stable Diffusion

Stable Diffusion generates images based on the prompt provided, and while it creates reliable results, it struggles to get the details right, irrespective of how detailed the prompt gets.

For this, VAE (Variational Autoencoder) models serve a great purpose to enhance the overall quality while improving the details, particularly facial structures and backgrounds. In this guide, we will help you get started with the VAE model by downloading and using them on Stable Diffusion with ease.

VAE in Stable Diffusion

  1. Start by navigating to Hugging Face on your browser and sign in with your credentials. Once you’re on the home, navigate to the search bar and enter VAE.
Search for VAE model
  1. By doing so, you will find multiple VAE models and pick the model that fits your requirements based on the available results. However, if you’re unsure, download the VAE model from here.
Download VAE model from Hugging Face
  1. After you download the model file, move or copy it to the VAE – models folder in Stable Diffusion WebUI (stable-diffusion-webui\models\VAE).
Paste model file to VAE
  1. If you have done it right, you will find the model added to the VAE folder along with the Put VAE here text document.
VAE model pasted
  1. With the VAE model moved to its assigned folder, restart your device for the changes to save and reflect.
  2. After a successful reboot, open Stable Diffusion and then generate an image using txt2img. For demonstration, we generated the image of a woman and noticed inconsistencies around her eyes.
VAE - Image generated without VAE
  1. To address inconsistencies, open the Settings on Stable Diffusion WebUI. Next, navigate to the options on the left, click on Stable Diffusion, and then VAE.
  1. From the multiple settings that are displayed, click on the drop-down below SD VAE and select the VAE that you downloaded earlier.
VAE - Select VAE model from Settings
  1. The VAE model is now selected, and to test if it enhances details, we have generated another image of a woman and were able to get better results without inconsistencies around eyes or teeth.
VAE - Image generated with VAE

Here is a side-by-side comparison of two different images with and without VAE:


Wrapping Up

Congratulations on learning how to get reliable images without inconsistencies on Stable Diffusion!

VAE models do an exceptional job of getting the details right. However, it isn’t always the same case due to improper prompts and incompatibility with the checkpoint used. Hence, we suggest you download and use the right VAE while experimenting with the prompt.

How To Use Stable Diffusion for NSFW

How To Use LoRA Stable Diffusion

What Is a Negative Prompt in Stable Diffusion


We will be happy to hear your thoughts

Leave a reply

Enable registration in settings - general