The Core Idea: Learning to Reverse Destruction
Diffusion models progressively add Gaussian noise to an image during training until only pure noise remains. The model learns to reverse each step. At inference, you start with pure noise and reverse the process guided by a text prompt.
The Score Function Intuition
The model learns the score function — the gradient of the log probability density. Intuitively: given this noisy image, in which direction should I adjust pixels to make it look more like a real image?
Classifier-Free Guidance
The model trains simultaneously with and without text conditioning. At inference, you interpolate between these extremes — strongly conditioning on the text prompt while maintaining image realism. The guidance scale controls this balance.
Stable Diffusion 4 Improvements in 2026
SD4 introduces a unified architecture handling text-to-image, image-to-image, inpainting, and video generation. The 3B parameter transformer backbone delivers 40% better prompt adherence and dramatically improved text rendering.
Frequently Asked Questions
How do diffusion models work?
Diffusion models learn to reverse a gradual noising process. During training the model learns to remove noise step by step. During generation it starts from pure noise and iteratively denoises guided by a text prompt.
What is the difference between DALL-E and Stable Diffusion?
DALL-E 3 is closed-source by OpenAI. Stable Diffusion is open-source and can be run locally or fine-tuned. DALL-E 3 is easier to use; Stable Diffusion offers more flexibility and customisation.
Are diffusion models used for video?
Yes. Sora, Runway Gen-3, and Kling AI use diffusion-based architectures for video generation, applying the same denoising principles across temporal frames.