MLOps Blog

Image Processing Techniques That You Can Use in Machine Learning Projects

4 min

Gopal Singh Panwar

21st April, 2023

Computer Vision ML Model Development

Image processing is a method to perform operations on an image to extract information from it or enhance it. Digital image processing has a broad range of applications such as image restoration, medical imaging, remote sensing, image segmentation, etc. Every process requires a different technique.

In this article, we will be covering the top 6 image processing techniques for machine learning.

Image Restoration
Linear Filtering
Independent Component Analysis
Pixelation
Template Matching
Image Generation Technique (GAN)

1. Image restoration

An image deteriorates for many reasons, for example, an old image of your grandparents which was taken with the old tech camera could become hazy or may lose its original form.

This could happen if the image goes under some physical stress or if it’s in digital form it could deteriorate by motion blur or additive noise.

So how are you going to restore it? Maybe it wasn’t possible 50 years back but now – it is.

Researchers came up with a Degradation model that can undo the deterioration effects on the input image. The degradation model works as a convolution with a linear shift-invariant.

So we take an Image before the degradation which is called “True Image” and an Image after degradation which is called “Observed Image” with the degradation filter which estimates the “True Image”.

An example of image restoration using image inpainting with OpenCV

Image impainting also known as “Compensation of paint loss ”. This technique is often used to remove unwanted objects from an image to restore damaged parts of a deteriorated image.

import cv2

img = cv2.imread('damaged_image.png')
mask = cv2.imread('mask.png', 0)
dst = cv2.inpaint(img, mask, 3, cv2.INPAINT_NS)

cv2.imwrite('restored.png', dst)

In the above code, we have two types of images

Damaged
Masked

A masked image has the same spatial dimensions of the noise which exists in the noisy image.

So if we input the image below with the above code:

With the mask:

Then we will get the following image:

The biggest problem with OpenCV’s image inpainting is that we need to manually input a mask for the specific image we want to fix. So how can we automate this process?

The answer is GAN (General Adversarial Network). This paper proposes that, by using a GAN network, image inpainting can be done using neighborhood loss function and gradient loss with a better quality restored image.

2. Linear filtering

Linear filtering is a process in which the value of the output pixel is linear combinations of the neighboring input pixels. This process is done by a technique called Convolution.

Convolution is the process of adding each element of the image to its local neighbors, weighted by the kernel.

We have an input image and a kernel with an anchor point. In the above diagram, it’s H(1, 1).

This filter works as a sliding window to convolve over the image.

We multiply each pixel by the corresponding kernel and then take the sum. That sum becomes a new pixel in the output image.

Let’s see this in action with the help of OpenCV

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt

image = cv.imread("pics/goku.jpeg")

fig, ax = plt.subplots(1, 3, figsize=(16, 8))
fig.tight_layout()

# To conovolve the kernel on an image we can use cv.filter2D
ax[0].imshow(cv.cvtColor(image, cv.COLOR_BGR2RGB))
ax[0].set_title('Original Image')

kernel_sharpening = np.array([[-1, -1, -1],
                             [-1, 9, -1],
                             [-1, -1, -1]])

kernel_sharpening_2 = np.array([[-1, -1, -1],
                             [-1, 10, -1],
                             [-1, -1, -1]])

sharpened = cv.filter2D(image, -1, kernel_sharpening)
ax[1].imshow(cv.cvtColor(sharpened, cv.COLOR_BGR2RGB))
ax[1].set_title('Sharpened Kernel Image')

sharpened_2 = cv.filter2D(image, -1, kernel_sharpening_2)
ax[2].imshow(cv.cvtColor(sharpened_2, cv.COLOR_BGR2RGB))
ax[2].set_title('Sharpened Kernel Image 2')

plt.show()

3. Independent Component Analysis

Independent Component Analysis or short for ICA is a technique for separating a multivariate signal into its underlying component. ICA helps in the extraction of the desired component from the mixture of multiple components or signals.

Let me explain.

In ICA, we “Whiten” our signal. This means that a given will be transformed in a way that potential correlations between its component are removed and the variance of each component is equal to 1.

ICA using sklearn

import numpy as np
import matplotlib.pyplot as plt
from scipy import signal
from sklearn.decomposition import FastICA, PCA

n_samples = 2000
time = np.linspace(0, 8, n_samples)

s1 = np.sin(2 * time)  # Signal 1 : sinusoidal signal
s2 = np.sign(np.sin(3 * time))  # Signal 2 : square signal
s3 = signal.sawtooth(2 * np.pi * time)  # Signal 3: saw tooth signal

S = np.c_[s1, s2, s3]
S += 0.2 * np.random.normal(size=S.shape)  # Add noise

S /= S.std(axis=0)  # Standardize data
# Mix data
A = np.array([[1, 1, 1], [0.5, 2, 1.0], [1.5, 1.0, 2.0]])  # Mixing matrix
X = np.dot(S, A.T)  # Generate observations

# Compute ICA
ica = FastICA(n_components=3)
S_ = ica.fit_transform(X)  # Reconstruct signals
A_ = ica.mixing_  # Get estimated mixing matrix

plt.figure()

models = [X, S, S_]
names = ['Observations (mixed signal)',
         'True Sources',
         'ICA recovered signals']
colors = ['red', 'steelblue', 'orange']

for ii, (model, name) in enumerate(zip(models, names), 1):
    plt.subplot(3, 1, ii)
    plt.title(name)
    for sig, color in zip(model.T, colors):
        plt.plot(sig, color=color)

plt.tight_layout()
plt.show()

Output:

4. Pixelation

Pixelation occurs when resizing of the images are enlarged to a point where individual pixels can be observed or pixels stretch to the point beyond their original size.

Pixelation using OpenCV

import cv2

# Input image
input = cv2.imread('cat.png')

# Get input size
height, width = input.shape[:2]

# Desired "pixelated" size
w, h = (16, 16)

# Resize input to "pixelated" size
temp = cv2.resize(input, (w, h), interpolation=cv2.INTER_LINEAR)

# Initialize output image
output = cv2.resize(temp, (width, height), interpolation=cv2.INTER_NEAREST)

cv2.imshow('Input', input)
cv2.imshow('Output', output)

cv2.waitKey(0)

Input:

Output:

5. Template matching

Template matching is a method for searching and finding the location of a template in a larger image. You can think of it as a very simple approach to object detection.

In template matching, we slide the template image over the larger image as we do in the convolution process and find the matching part

Template matching using OpenCV

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

img_rgb = cv.imread('waldo.jpg')
img_gray = cv.cvtColor(img_rgb, cv.COLOR_BGR2GRAY)
template = cv.imread('waldo_temp.jpg',0)
w, h = template.shape[::-1]
res = cv.matchTemplate(img_gray,template,cv.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where( res >= threshold)

for pt in zip(*loc[::-1]):
    cv.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)

cv.imshow('img_rgb', img_rgb)
cv.waitKey(0)

Template:

Larger image:

Output:

6. Image Generation Technique (GAN)

With the help of the Generative Adversarial Networks (GANs), we can train a deep learning model on the image data to generate the same type of image data.

GANs were invented by Ian Goodfellow in 2014 which he described in the paper of Generative Adversarial Nets.

GANs are made of two distinct models

Generator
Discriminator

The job of the generator is to generate the fake images and discriminator try to classify between the fake image and real image. During the training, the generator tries to outsmart the discriminator by generating better fake images and the discriminator tries to improve itself for differentiating between the real image and a fake image. You can read more about GAN architectures and training in this article.