Before we begin, we highly recommend commissioning art from an ACTUAL artist who can draw the character you want, and can polish the work to be the one of your dreams. AI should only be used as a starting base, and for fun.

What is Pony Diffusion XL?

In a nutshell, PonyXL is an advanced text-to-image AI model that uses deep learning techniques to generate highly detailed and creative images from textual descriptions. It was mainly trained to… well My Little Pony characters… But, during the training process, PonyXL’s ability to generate quality art in various styles as well. While it first began as a way for Furry community to generate their Fursona, people quickly noticed how PonyXL was able to distinguish sources between Anime, Cartoon, 3D, Pony, and many more. 

PonyXL uses the tagging system below;

Positive Prompt; score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, just describe what you want, tag1, tag2

For Negative Prompts, you can use none, or you can delete Score 6, Score 5, and Score 4 from positive, and add it to negative prompts. One thing PonyXL doesn’t like is excessive prompting. It can understand even the most complicated prompts with the right wording.

For specific styles, you can add Source_furry, Source_cartoon, Source_pony, etc.

CLIP Skip; PonyXL model uses Clip Skip 2, which means on ComyUI, you will need to write -2 instead.

Sampler; Start with Eular A (Karras or Normal) at 25 steps with CFG at 7, and test out different samplers, with HIGHER number of steps and CFG value. If we want 3d aimed photos, we like to use DPM++ 3M SDE Karras (GPU) at 45 steps and 9 CFG, because rtx 4000 series graphics card seem to generate extremely good results. depending on the prompt, this setting may generate burnt images at steps higher than 50.

To keep the generations PG, make sure to add rating_safe or at the very least, source_questionable. If you do add source_explicit, you will only get results that may not be suitable for general audience. The last part can happen without the safe prompt added to the positives.

What is ComfyUI?

ComfyUI is an easy-to-use graphical interface designed for managing and interacting with AI models. It provides a user-friendly platform where people can load, run, and visualize the outputs of different AI models without needing to write code.

With ComfyUI, users can set up and control AI tasks through simple clicks and menus, making it accessible even for those who aren’t tech experts. It’s like having a control panel that simplifies working with complex AI tools, making advanced technology more approachable and practical for everyday use.

With ComfyUI, once you download the zip file for the Portable version from the GitHub page, you can simply run on your NVIDIA graphics card, or run on your CPU.

Just how easy is ComfyUI?

Luckily, every image generated by ComfyUI retains the meta data. This means you can download any of the images below, and simply drag to your ComfyUI tab, and it will recreate the same workflow we used to generate the image. From there, you can tweak the sampler, the prompts, the seed, etc. Have fun!

Setting up ComfyUI with your model

If you downloaded and dragged one of the images below, your set up or “WORKFLOW” will look something like this;

You downloaded PonyXL, it is sitting in your Downloads folder. That is no use to you. Close any ComfyUI tab that are open on your browser, close the Command Prompt that starts with ComfyUI, and go to ComfyUI folder, and go to Models. There, add your PonyXL, or any other models you may want to use under “Checkpoints” folder.

PonyXL comes with a VAE. In the same Models folder, there is a VAE folder. Don’t forget to download the recommended VAE file and put it in the correct folder. As you can see from the lower left corner in our screen shot, our VAE is chosen with the PONYXL model on the upper left corner.

You are now ready to start up ComfyUI again, and play around PonyXL. Have fun 🙂

Let us know about your generations by tagging us on Twitter with @WaifusAway and Instagram with @WaifuIsekai