PIL stands for Python Image Library. In this article, we will look at its fork: Pillow. PIL has not been updated since 2011 and so the case for Pillow is obvious.
The library provides support for various image formats including the popular JPEG and PNG formats. Another reason you would consider using Pillow is the fact that it is quite easy to use and very popular with Pythonistas. The package is a common tool in the arsenal of most data scientists who work with images.
It also provides various image processing methods as we will see in this piece. These techniques are very useful especially in augmenting training data for computer vision problems.
Here is what you can expect to learn:
- How to install it
- Essential Pillow concepts
- The Pillow Image Class
- Reading and writing images with PIL/Pillow
- Pillow image manipulation e.g cropping and rotating images
- Image enhancement with Pillow e.g filters to improve image quality
- Working with image sequences (GIFs) in PIL
MIGHT BE USEFUL
Best Image Processing Tools used in Machine Learning
Notes on installation
Pillow can be installed via pip.
pip install Pillow
It is important to note that the Pillow and PIL packages can not coexist in the same environment. Therefore ensure that you do not have PIL installed as you install Pillow.
Now grab your pillow and let’s get started.
Essential PIL image concepts
We will start by understanding a couple of PIL image concepts that are critical.
In order to see these concepts in action, let’s start by loading in this image using Pillow. The first step is to import the
Image class from PIL. We will talk more about the
Image class in the next part.
We import the Image class from PIL not Pillow because Pillow is a PIL fork. Hence moving forward you should expect imports from PIL and not Pillow.
from PIL import Image im = Image.open("peacock.png")
With the image loaded, Let’s start talking about those image concepts.
Every image has one or more bands. Using Pillow we can store one or several bands in an image. For example, a colored image will usually have ‘R’, ‘G’, and ‘B’ bands for Red, Blue, and Green respectively.
Here is how we can obtain the bands for the image we imported above.
im.getbands() ('R', 'G', 'B')
Mode refers to the type and depth of a pixel in an image. Some of the modes that are currently supported are:
- L for black and white
- RGB for true color
- RGBA for true color with transparency mask
- YCbCr for color video format
Here is how we would obtain the mode for the image we loaded above:
We can also obtain the dimensions of an image via the image attribute.
The image loaded above was quite large, so I reduced the size so that it might be easier to visualize in later parts of this article.
im.size (400, 600)
The Pillow package uses the Cartesian pixel coordinate system. In later parts of this piece, we will use this concept, so it is paramount that you understand it. In this system:
- (0,0) is the upper left corner
- coordinates are passed as a tuple in the form (x,y)
- rectangles are represented as 4 tuples, the upper left corner is provided first.
Understanding the Image class
As we have mentioned earlier we had to import the
Image class from PIL before reading in our image. This class contains functions that enable us to load image files as well as to create new images. Moving forward, the functions we will use have been imported as a result of importing
Image, unless otherwise stated.
Loading and saving images
We have already seen that we can load in an image using
peacock.jpg is the path to the location of the image.
Reading from a String
In order to demonstrate how to read in an image string using Pillow, we will start by converting the image to a string via base64.
import base64 with open("peacock.jpg", "rb") as image: image_string = base64.b64encode(image.read())
We can now decode the image string and load it as an image using the
Image class from PIL.
import io image = io.BytesIO(base64.b64decode(image_string)) Image.open(image)
Convert to JPEG
Let’s now take an example of how we can convert an image to the JPEG format.
PIL Save Image
Image conversion is achieved by reading in the image and saving it with the new format. Here is how we would convert the peacock PNG image to JPEG format.
Create JPEG thumbnails
In some cases, we would be interested in reducing the size of an image.
For instance, one can reduce the size of an image in order to obtain the image’s thumbnails. This can be accomplished by defining the size of the thumbnail and passing it to the
thumbnail image function.
size = 128, 128 im.thumbnail(size) im.save('thumb.png')
We have already seen that we can manipulate various aspects of an image such as size. Let’s dive in a little deeper and look at other aspects such as image rotation and color transformations – just to mention a couple.
In order to crop an image, we start by defining a box that dictates the area of the image that we would like to crop. Next, we pass that box to the `crop` function of the
im = Image.open('peacock.jpg') box = (100, 150, 300, 300) cropped_image = im.crop(box) cropped_image
Rotating an image is done via the
rotate function of the
rotated = im.rotate(180) rotated
The package also allows us to merge two images. Let’s illustrate that by merging a logo into the peacock image. We start by importing it.
logo = Image.open('logo.png')
Let’s now define the position of the logo and merge the two images.
position = (38, 469) im.paste(logo, position) im.save('merged.jpg')
This operation takes place in place of the original image. Therefore, if you want to keep the original image, you can make a copy of it.
image_copy = image.copy()
Now if you are using a PNG image you can take advantage of Pillow’s masking capability to get rid of the black background.
im = Image.open("peacock.jpg") image_copy = im.copy() position = ((image_copy.width - logo.width), (image_copy.height - logo.height)) image_copy.paste(logo, position,logo) image_copy
Let’s now look at how we can flip the above image. This is done using the “Flip” method. Some of the flip options are
PIL image to NumPy array
Pillow also allows us to convert an image to a NumPy array. After converting an image to NumPy array we can read it in using PIL.
import numpy as np im_array = np.array(im)
With the image converted we can now load it using Pillow. This is done using the
fromarray function of Pillow’s Image class. Finally, we save and display the image using PIL
show image function.
img = Image.fromarray(im_array, 'RGB') img.save('image.png') img.show()
We can switch an image from colored to black and white and vice versa. This is done via the convert function and passing the preferred color format.
Converting to colored can be done in a similar manner.
Drawing on images
We are about to wrap it up, before we do let’s look at a couple more items, including drawing on the image. Pillow allows that to be done via the
ImageDraw module, we, therefore, start by importing it.
from PIL import ImageDraw
we will start by defining a blank colored image of size 400 by 400. We then use
ImageDraw to draw the image.
image = Image.new('RGB', (400, 400)) img_draw = ImageDraw.Draw(image)
Now we can use the
ImageDraw object to draw a rectangle on the image. We fill it with the white color and give a red outline. Using the same object we can write some text on the image as shown.
img_draw.rectangle((100, 30, 300, 200), outline='red', fill='white') img_draw.text((150, 100), 'Neptune AI', fill='red') image.save('drawing.jpg')
Pillow also comes with functions that enable us to perform image enhancement. This is a process that improves the original quality of an image.
We start by importing the module that ships those functionalities.
from PIL import ImageEnhance
For example, we can adjust the image sharpness:
from PIL import ImageEnhance enhancer = ImageEnhance.Sharpness(im) enhancer.enhance(10.0)
Let’s take another example where we double the brightness of the image.
enhancer = ImageEnhance.Contrast(im) enhancer.enhance(2)
Another super cool thing we can do with Pillow is to add filters to images. The first step is to import the
from PIL import ImageFilter
For instance, we can blur the image like so:
from PIL import ImageFilter im = Image.open("peacock.jpg") im.filter(ImageFilter.BLUR)
Other filters that are available include:
Working with images sequences (GIFs) in PIL
We can also load image sequences such as GIF images. Let’s start by importing the image sequence module.
from PIL import ImageSequence
Next, we will load in a GIF and save the first two frames as a PNG file. We break out of the loop because the frames are too many.
im = Image.open("embeddings.GIF") frame_num = 1 for frame in ImageSequence.Iterator(im): frame.save("frame%d.png" % frame_num) frame_num = frame_num + 1 if frame_num == 3: break
Let’s look at one of the frames that has been saved as a PNG file.
Hopefully this piece has given you some insights on how you can apply Pillow in your image processing pipeline.
An example of such a use case is in image augmentation sequences that you use in image segmentation problems. This would help you in creating more image instances which eventually improves the performance of your image segmentation model.
Let’s now recap on some of the things we covered:
- the process of installing PIL/Pillow
- fundamental Pillow concepts
- using the Image class in PIL/Pillow
- reading and writing images in PIL/Pillow
- manipulating images in Pillow
- improving image quality using Pillow
- adding filters to Images
- reading image sequences (GIFs)
- converting a Pillow image to a NumPy
You can start applying the skills you have learned in this article right away. One good application example is in image augmentation for deep learning models. Synthetically increasing the quantity of your training data is one of the best (and simplest) ways to improve your model’s performance.
Just try it out!
ML Experiment Tracking: What It Is, Why It Matters, and How to Implement It
Jakub Czakon | Posted November 26, 2020
Let me share a story that I’ve heard too many times.
”… We were developing an ML model with my team, we ran a lot of experiments and got promising results…
…unfortunately, we couldn’t tell exactly what performed best because we forgot to save some model parameters and dataset versions…
…after a few weeks, we weren’t even sure what we have actually tried and we needed to re-run pretty much everything”
– unfortunate ML researcher.
And the truth is, when you develop ML models you will run a lot of experiments.
Those experiments may:
- use different models and model hyperparameters
- use different training or evaluation data,
- run different code (including this small change that you wanted to test quickly)
- run the same code in a different environment (not knowing which PyTorch or Tensorflow version was installed)
And as a result, they can produce completely different evaluation metrics.
Keeping track of all that information can very quickly become really hard. Especially if you want to organize and compare those experiments and feel confident that you know which setup produced the best result.
This is where ML experiment tracking comes in.Continue reading ->