5. Object Detection and Image Segmentation › 5.4 Image Segmentation

Section 5.4
Image Segmentation

Example of three different types of segmentation tasks
Figure 64: Example of three different types of segmentation tasks.

Image segmentation falls under the umbrella of pixel labeling, where every pixel in an image is assigned a label.1 Contrast this to image classification (from the last chapter) where the entire image was assigned a single label. Pixel labels can be anything from (continuous) depth to abstract identifiers to (discrete) semantic category and combinations thereof. In image segmentation tasks include unsupervised segmentation, which breaks an image up into meaningful regions or superpixels and tends to be driven by low-level visual cues, semantic segmentation, which label each pixel with as belonging to a given class, and instance segmentation where each pixel is assigned an object/instance identifier. Compared to bounding box object detection, instance segmentation can provide a precise outline of each object. Examples of the different segmentation tasks are shown in Figure 64.2

  1. 1. Sometimes you will also hear the term dense prediction, which is essentially the same thing.
  2. 2. While the Segment Anything model (SAM) [57] is a foundation model, which can be prompted to produce different segmentation types, the example shown in Figure 64(a) is typical of output that you would see from an unsupervised segmentation algorithm for that scene.