# Some more Variational Image Processing: Diffusion, TV Denoising, TV Image Inpainting in Python

In this article, a few variational image processing techniques will  be described along with application of those techniques with some images, most of the problems are taken from the assignments from this course.

## Some preliminaries: The Calculus of Images – Computing Curvature and TV

• Let’s first compute the Euclidian Curvature of a few images.  The curvature measures the rate at which the unit gradient vector is changing and is given by
• The following couple of images are used to compute the curvature. As can be seen from the below figures, the curvature is zero in flat regions and along straight edges and non-zero along the rounded edges of the circles, as expected.
• Now, let’s compute the total variation (TV), which is given by the following.
• First we need to approximate the partial derivatives using a forward difference.
• Let’s compute the TV for the grayscale image Cameraman. Now let’s add more and more Salt & Pepper noise (by increasing the probability threshold p) to the image and see how the norm of the gradient matrix along with the TV value changes from the following figure.

## The Heat Equation and Diffusion

Let’s implement the isotropic and anisotropic diffusion by solving PDEs numerically.
The following figure shows the math.

The following shows the isotropic diffusion output with Δt = 0.1, with gradient descent.  As can be seen, the results are same as applying gaussian blur kernel on the image.

The following shows the anisotropic diffusion output with Δt = 0.1, with gradient descent, with a = 5, 20, 100 respectively.  As can be seen, unlike isotropic diffusion, the anisotropic diffusion preserves the edges.

## Creating Cartoon / flat-texture Images with Anisotropic Diffusion

As can be seen from the following figures and animations, we can create cartoons from the real images with anisotropic diffusion, by applying the diffusion on each channel, this time on color images (the barbara image and my image).

Original image

Cartoon Image with anisotropic diffusion (a=5)

Original Image

Cartoon Image with anisotropic diffusion (a=5)

## Total Variation Denoising

The following math is going to be used for TV denoising, the Euler-Lagrange equation is used to solve the minimum of the functional, as shown in the following figures with proof.

• First a noisy grayscale image is prepared by adding Gaussian noise to the cameraman image.

Original Cameraman

Noisy Cameraman

• Let’s first denoise this image with linear TV denoising. The next animations show the results obtained , using the fidelity weight λ=1. As can be seen, even with the fidelity term, this model blurs the edges.
• Now let’s denoise this image with Nonlinear TV denoising. The next animations show the results obtained , using the fidelity weight λ=0.01 and λ=1 respectively.

## Image Inpainting

Inpainting is the process of restoring damaged or missing parts of an image. Suppose we have a binary mask D that specifies the location of the damaged pixels in the input image f as shown below:

The following theory is going to be used for TV inpainting.

Damaged image

Damaged image

Damaged image

Damaged image

## 4 thoughts on “Some more Variational Image Processing: Diffusion, TV Denoising, TV Image Inpainting in Python”

1. Royi Avital

Hi,

Nice post.
You better use Animated PNG (APNG) instead of GIF for better quality of your animations.

Like

• Thanks for the suggestion Royi

Like