Chapter 11
Generative AI: VAEs, Diffusion and Flows
This lecture presents models that sample from complicated probability distributions even if we cannot write down the distribution itself. We focus primarily on generating images since they are a good example of objects with complex structure and intractable distributions. They were also the first domain in which many of these models were developed. We start variational auto-encoders, which establish a baseline for the more sophisticated flow and diffusion models presented later in the lecture. Students will benefit from a background in differential equations, probability theory and probabilistic graphical models.
The models studies thus far in the course have been about making predictions—whether it’s predicting a label for an image, the next word in a sentence, or the most rewarding action to take. We now turn our attention to generative AI, i.e., methods for drawing samples from a probability distribution. The distribution may be high-dimensional and complex, such as the distribution of natural images. Worse, we are not even given the distribution but a (small) set of samples from it. Our aim is not necessarily to recover an explicit form for the unknown underlying distribution. It is sufficient for us to generate novel and diverse samples that are in some sense similar to the set that we are given.
The big picture idea of generative AI is to learn a mapping from distributions that are easy to sample from, such as multivariate Gaussians, to unknown or hard-to-sample distributions. This is illustrated in Figure 131.
