What Is ControlNet Stable Diffusion

Are you not happy with the img2img model enhancement on Stable Diffusion? Learn how to do it better with ease through the steps below.
What Is ControlNet Stable Diffusion

ControlNet is paired with Stable Diffusion to enhance the generation of images with more control based on input conditions or constraints. In this guide, we will help you get more precise images with steps and a demonstration to get you all covered.

ControlNet on Stable Diffusion     

  1. Open a new tab on your browser and open Mikubill’s ControlNet GitHub page. After it is loaded, click on the Code button and then the Copy button next to copy the URL under HTTPS.   
ControlNet - Copy clone URL
  1. Start by opening Stable Diffusion WebUI on your device. Once it is loaded, click on Extensions and then Install from URL.
ControlNet - Click on Extensions
  1. Now, paste the copied URL in the first input box and click the Install button to add ControlNet to Stable Diffusion.
ControlNet - Add clone URL
  1. After the ControlNet is installed successfully, download Illyasviel’s ControlNet from Hugging Face or CivitAI.
ControlNet -Model from Hugging face
  1. If you’re confused about choosing a version from the vast list, download the ControlNet version that worked for us from here.If you’re confused about choosing a version from the vast list, download the ControlNet version that worked for us from here.
ControlNet - Download model
  1. Wait for the file download as it is significant, and once done, move or copy and paste it to the stable-diffusion-webui\extensions\sd-webui-controlnet\models folder.
ControlNet - Paste the downloaded ControlNet model
  1. While it isn’t mandatory, we suggest you restart your device to make the changes appear. Next, open Stable Diffusion. When you scroll down to the bottom and in the ControlNet section, select Canny as the Control Type and check Enable and Pixel Perfect.
ControlNet - Select Canny, Enable, Pixel Perfect and Upload image
  1. With specified options set, upload the image, navigate to the top to enter the positive and negative prompts, and then click on the Generate button.

Quick Tip: You can set options under Generation as default. However, if you have a specific dimension requirement, adjust the Height and Width. Also, experiment with the CFG scale by adjusting it according to the image generated.

ControlNet - Add prompt
  1. Now, wait for a while, as image generation can take a while. Once you are done, you will find the preview of the image generated with ControlNet on the right.
ControlNet - Image generated

Here is the comparison of the image added as a reference and ControlNet’s result:

Comparison - ControlNet

    Wrapping Up

    Congratulations on learning how to enhance images with ControlNet on Stable Diffusion!

    While the result we generated matched our expectations, if you’re not happy with your results, try writing better prompts and adjust the CFG scale between 7-13. However, if nothing works, verify if the ControlNet downloaded is compatible with the checkpoint you’re using.

    How To Use img2img Stable Diffusion

    What Is VAE Stable Diffusion

    How To Use Stable Diffusion for NSFW

    How To Use LoRA Stable Diffusion


    We will be happy to hear your thoughts

    Leave a reply

    Enable registration in settings - general