Essential Pil (Pillow) Image Tutorial (for Machine Learning People)

Posted October 13, 2020

PIL stands for Python Image Library. In this article, we will look at its fork: Pillow. PIL has not been updated since 2011 and so the case for Pillow is obvious. 

The library provides support for various image formats including the popular JPEG and PNG formats. Another reason you would consider using Pillow is the fact that it is quite easy to use and very popular with Pythonistas. The package is a common tool in the arsenal of most data scientists who work with images. 

It also provides various image processing methods as we will see in this piece. These techniques are very useful especially in augmenting training data for computer vision problems. 

 Here is what you can expect to learn: 

  • How to install it 
  • Essential Pillow concepts
  • The Pillow Image Class
  • Reading and writing images with PIL/Pillow
  • Pillow image manipulation e.g cropping and rotating images
  • Image enhancement with Pillow e.g filters to improve image quality
  • Working with image sequences (GIFs) in PIL

See also:

Best Image Processing Tools used in Machine Learning


Let’s go! 

Notes on installation 

Pillow can be installed via pip. 

pip install Pillow

It is important to note that the Pillow and PIL packages can not coexist in the same environment. Therefore ensure that you do not have PIL installed as you install Pillow. 

Now grab your pillow and let’s get started. 

Essential PIL image concepts

We will start by understanding a couple of PIL image concepts that are critical. 

In order to see these concepts in action, let’s start by loading in this image using Pillow. The first step is to import the Image class from PIL. We will talk more about the Image class in the next part. 

Note:

We import the Image class from PIL not Pillow because Pillow is a PIL fork. Hence moving forward you should expect imports from PIL and not Pillow.

from PIL import Image
im = Image.open("peacock.png")

With the image loaded, Let’s start talking about those image concepts. 

Bands

Every image has one or more bands. Using Pillow we can store one or several bands in an image. For example, a colored image will usually have ‘R’, ‘G’, and ‘B’ bands for Red, Blue, and Green respectively. 

Here is how we can obtain the bands for the image we imported above. 

im.getbands()
('R', 'G', 'B')

Modes

Mode refers to the type and depth of a pixel in an image. Some of the modes that are currently supported are: 

  • L for black and white 
  • RGB for true color 
  • RGBA for true color with transparency mask 
  • YCbCr for color video format

Here is how we would obtain the mode for the image we loaded above: 

im.mode
'RGB'

Size 

We can also obtain the dimensions of an image via the image attribute. 

Note:

The image loaded above was quite large, so I reduced the size so that it might be easier to visualize in later parts of this article.

im.size
(400, 600)

Coordinate system

The Pillow package uses the Cartesian pixel coordinate system. In later parts of this piece, we will use this concept, so it is paramount that you understand it. In this system:

  • (0,0) is the upper left corner
  • coordinates are passed as a tuple in the form (x,y)
  • rectangles are represented as 4 tuples, the upper left corner is provided first. 

Understanding the Image class

As we have earlier we had to import the Image class from PIL before reading in our image. This class contains functions that enable us to load image files as well as to create new images. Moving forward, the functions we will use have been imported as a result of importing Image, unless otherwise stated. 

Loading and saving images 

We have already seen that we can load in an image using Image.open("peacock.jpg") where peacock.jpg is the path to the location of the image. 

Reading from a String

In order to demonstrate how to read in an image string using Pillow, we will start by converting the image to a string via base64

import base64

with open("peacock.jpg", "rb") as image:
    image_string = base64.b64encode(image.read())

We can now decode the image string and load it as an image using the Image class from PIL. 

import io

image = io.BytesIO(base64.b64decode(image_string))
Image.open(image)
pil/pillow peacock image

Convert to JPEG

Let’s now take an example of how we can convert an image to the JPEG format. 

PIL Save Image

Image conversion is achieved by reading in the image and saving it with the new format. Here is how we would convert the peacock PNG image to JPEG format. 

im.save('peacock.jpg')

Create JPEG thumbnails

In some cases, we would be interested in reducing the size of an image. 

For instance, one can reduce the size of an image in order to obtain the image’s thumbnails. This can be accomplished by defining the size of the thumbnail and passing it to the thumbnail image function.

size = 128, 128
im.thumbnail(size)
im.save('thumb.png')
pil thumbnail

Image manipulation 

We have already seen that we can manipulate various aspects of an image such as size. Let’s dive in a little deeper and look at other aspects such as image rotation and color transformations – just to mention a couple. 

Cropping images

In order to crop an image, we start by defining a box that dictates the area of the image that we would like to crop. Next, we pass that box to the `crop` function of the Image class.

im = Image.open('peacock.jpg')
box = (100, 150, 300, 300)
cropped_image = im.crop(box)
cropped_image
pil cropped image

Rotating images

Rotating an image is done via the rotate function of the Image class.

rotated = im.rotate(180)
rotated
pil image rotated

Merging images

The package also allows us to merge two images. Let’s illustrate that by merging a logo into the peacock image. We start by importing it. 

logo = Image.open('logo.png')

Let’s now define the position of the logo and merge the two images. 

position = (38, 469)
im.paste(logo, position)
im.save('merged.jpg')

This operation takes place in place of the original image. Therefore, if you want to keep the original image, you can make a copy of it. 

image_copy = image.copy()
pil merging images

Now if you are using a PNG image you can take advantage of Pillow’s masking capability to get rid of the black background.

im = Image.open("peacock.jpg")
image_copy = im.copy()
position = ((image_copy.width - logo.width), (image_copy.height - logo.height))
image_copy.paste(logo, position,logo)
image_copy
pil merging images

Flip images

Let’s now look at how we can flip the above image. This is done using the “Flip” method. Some of the flip options are FLIP_TOP_BOTTOM and FLIP_LEFT_RIGHT.

im.transpose(Image.FLIP_TOP_BOTTOM)
pil transpose image

PIL image to NumPy array

Pillow also allows us to convert an image to a NumPy array. After converting an image to NumPy array we can read it in using PIL. 

import numpy as np
im_array = np.array(im)

With the image converted we can now load it using Pillow. This is done using the fromarray function of Pillow’s Image class. Finally, we save and display the image using PIL show image function. 

img = Image.fromarray(im_array, 'RGB')
img.save('image.png')
img.show()

Color transformations

We can switch an image from colored to black and white and vice versa. This is done via the convert function and passing the preferred color format. 

im.convert('L')
pil no colors

Converting to colored can be done in a similar manner. 

im.convert('RGBA')
pil colors back

Drawing on images

We are about to wrap it up, before we do let’s look at a couple more items, including drawing on the image. Pillow allows that to be done via the ImageDraw module, we, therefore, start by importing it.

from PIL import  ImageDraw

we will start by defining a blank colored image of size 400 by 400. We then use ImageDraw to draw the image. 

image = Image.new('RGB', (400, 400))
img_draw = ImageDraw.Draw(image)

Now we can use the ImageDraw object to draw a rectangle on the image. We fill it with the white color and give a red outline. Using the same object we can write some text on the image as shown. 

img_draw.rectangle((100, 30, 300, 200), outline='red', fill='white')
img_draw.text((150, 100), 'Neptune AI', fill='red')
image.save('drawing.jpg')
pil image draw

Image enhancement

Pillow also comes with functions that enable us to perform image enhancement. This is a process that improves the original quality of an image. 

We start by importing the module that ships those functionalities. 

from PIL import ImageEnhance

For example, we can adjust the image sharpness:

from PIL import ImageEnhance
enhancer = ImageEnhance.Sharpness(im)
enhancer.enhance(10.0)
pil sharpness

Let’s take another example where we double the brightness of the image. 

enhancer = ImageEnhance.Contrast(im)
enhancer.enhance(2)
pil brightness

Filters

Another super cool thing we can do with Pillow is to add filters to images. The first step is to import the ImageFilter module. 

from PIL import ImageFilter

For instance, we can blur the image like so:

from PIL import ImageFilter
im = Image.open("peacock.jpg")

im.filter(ImageFilter.BLUR)
pil blur image

Other filters that are available include:

  • CONTOUR
im.filter(ImageFilter.CONTOUR)
pil contour
  • DETAIL
im.filter(ImageFilter.DETAIL)
pil image detail
  • EDGE_ENHANCE
im.filter(ImageFilter.EDGE_ENHANCE)
pil egde enhance
  • EMBOSS
im.filter(ImageFilter.EMBOSS)
pil emboss
  • FIND_EDGES
im.filter(ImageFilter.FIND_EDGES)
pil find edges

Working with images sequences (GIFs) in PIL

We can also load image sequences such as GIF images. Let’s start by importing the image sequence module. 

from PIL import ImageSequence

Next, we will load in a GIF and save the first two frames as a PNG file. We break out of the loop because the frames are too many. 

im = Image.open("embeddings.GIF")

frame_num = 1
for frame in ImageSequence.Iterator(im):
    frame.save("frame%d.png" % frame_num)
    frame_num = frame_num + 1
    if frame_num == 3:
        break

Let’s look at one of the frames that has been saved as a PNG file. 

pil image sequences frame

Final thoughts

Hopefully this piece has given you some insights on how you can apply Pillow in your image processing pipeline. 

An example of such a use case is in image augmentation sequences that you use in image segmentation problems. This would help you in creating more image instances which eventually improves the performance of your image segmentation model. 

Let’s now recap on some of the things we covered:

  • the process of installing PIL/Pillow
  • fundamental Pillow concepts
  • using the Image class in PIL/Pillow
  • reading and writing images in PIL/Pillow
  • manipulating images in Pillow 
  • improving image quality using Pillow
  • adding filters to Images
  • reading image sequences (GIFs) 
  • converting a Pillow image to a NumPy 

You can start applying the skills you have learned in this article right away. One good application example is in image augmentation for deep learning models. Synthetically increasing the quantity of your training data is one of the best (and simplest) ways to improve your model’s performance. 

Just try it out!

Data Scientist