Stable Diffusion prompt tutorial: Basics of prompt engineering for Stable Diffusion

What is Prompt Engineering?

Prompt engineering is a concept in artificial intelligence, particularly natural language processing (NLP). In prompt engineering, the description of the task is embedded in the input. The output of prompts might differ, regarding tool and model you choose. For example, different results you will get from different Stable Diffusion models.

Raw prompts?

A raw prompt is a very basic description of what you want to generate. Most people start by using only raw prompts. It is a common mistake because these images tend to get random and chaotic. The results are OK but but we can do better :) As an example I generated a Tiger a skyscraper and a watch with stable diffusion.

As you can see there is nothing special about these images, it could be a google image search as well :)

Adding style to our prompt

The style of the prompt is very important. If the AI is missing a specific style, it will usually choose the one it has seen the most in similar images. Having a well-chosen style and raw prompt is sometimes enough, as the style influences the image the most right after the raw prompt.

The most commonly used styles include:

1. Realistic
2. Oil painting
3. Pencil drawing
4. Concept art
5. digital art
6. Cartoon
7. Anime
8. Manga
9. Sketch
10. Watercolor
11. Abstract
12. Surrealism
13. Pop art
14. Futurism

We can see the result is getting better with the style addition, Tiger as a pencil drawing is a good example.

In the case of a realistic image, there are various ways of making it the best. Here are some commonly used techniques of making the image realistic:

1. photo of + raw prompt
2. photograph of + raw prompt
3. raw prompt, hyperrealistic
4. raw prompt, realistic

If you want your prompt to generate an oil painting, you can add "an oil painting of" to your prompt. This may sometimes result in the image showing an oil painting in a frame. If this happens, you can just re-run the prompt or use raw prompt + "oil painting." To create a pencil drawing, add "a pencil drawing of" to your raw prompt, or make your prompt raw prompt + "pencil drawing."

Adding an artist to your prompt

To make your style more specific, you can use artists’ names in your prompt. For instance, if you want a very abstract image, you can add “made by Pablo Picasso” or just simply, “Picasso”. Below are lists of artists in different styles that you can use, but I always encourage you to search for different artists as it is a cool way of discovering new art.

Portrait

John Singer Sargent, Edgar Degas, Paul Cézanne, Jan van Eyck

Oil painting

Leonardo DaVinci, Vincent Van Gogh, Johannes Vermeer, Rembrandt

Pencil/Pen drawing

Albrecht Dürer, Leonardo da Vinci, Michelangelo, Jean-Auguste-Dominique Ingres

Landscape art

Thomas Moran, Claude Monet, Alfred Bierstadt, Frederic Edwin Church

Tiger Oil Painting made by Pablo Picasso

Finishing touches

Some final things you can add to your prompt to make it look more like what you want include adding "trending on artstation" for a more artistic image, or "Unreal Engine" for more realistic lighting. You can add anything you want, but these are some examples: Highly detailed, surrealism, trending on art station, triadic color scheme, smooth, sharp focus, matte, elegant, the most beautiful image ever seen, illustration, digital paint, dark, gloomy, octane render, 8k, 4k, washed colors, sharp, dramatic lighting, beautiful, post processing, picture of the day, ambient lighting, epic composition, bokeh, 15mm wide lens, 35mm wide lens, 85mm wide lens,

Image below is a "tiger toy made of yarn"

More resources

For creating samples I used stable diffusion model on https://beta.dreamstudio.ai
To search art with prompts https://lexica.art/

Final words

Hope you enjoyed this simple guide! Thank you! If you enjoyed this tutorial you can find more and continue reading on our tutorial page