Some more Variational Image Processing: Diffusion, TV Denoising, TV Image Inpainting in Python

In this article, a few variational image processing techniques will  be described along with application of those techniques with some images, most of the problems are taken from the assignments from this course.

 

Some preliminaries: The Calculus of Images – Computing Curvature and TV

 

  • Let’s first compute the Euclidian Curvature of a few images.  The curvature measures the rate at which the unit gradient vector is changing and is given byf11.png
  • The following couple of images are used to compute the curvature. As can be seen from the below figures, the curvature is zero in flat regions and along straight edges and non-zero along the rounded edges of the circles, as expected.cur_testcur_source
  • Now, let’s compute the total variation (TV), which is given by the following.f12.png
  • First we need to approximate the partial derivatives using a forward difference.
  • Let’s compute the TV for the grayscale image Cameraman. Now let’s add more and more Salt & Pepper noise (by increasing the probability threshold p) to the image and see how the norm of the gradient matrix along with the TV value changes from the following figure.cam
    tv_Cameraman256.png

The Heat Equation and Diffusion

Let’s implement the isotropic and anisotropic diffusion by solving PDEs numerically.
The following figure shows the math.

f13.png

The following shows the isotropic diffusion output with Δt = 0.1, with gradient descent.  As can be seen, the results are same as applying gaussian blur kernel on the image.

iso_cam.gif

The following shows the anisotropic diffusion output with Δt = 0.1, with gradient descent, with a = 5, 20, 100 respectively.  As can be seen, unlike isotropic diffusion, the anisotropic diffusion preserves the edges.

aniso_cam_5

aniso_cam_20

aniso_cam_100

Creating Cartoon / flat-texture Images with Anisotropic Diffusion

As can be seen from the following figures and animations, we can create cartoons from the real images with anisotropic diffusion, by applying the diffusion on each channel, this time on color images (the barbara image and my image).

Original image

barbara

Cartoon Image with anisotropic diffusion (a=5)

aniso_barbara_020.0_5

aniso_bar_col_10

Original Image

me2

Cartoon Image with anisotropic diffusion (a=5)

aniso_me2_020.0_5

 

 

me_col_7.5

Total Variation Denoising

The following math is going to be used for TV denoising, the Euler-Lagrange equation is used to solve the minimum of the functional, as shown in the following figures with proof.

f14.pngf9

f10.png

  • First a noisy grayscale image is prepared by adding Gaussian noise to the cameraman image.

 

Original Cameraman
Cameraman256

Noisy Cameraman

noisy_cam

 

  • Let’s first denoise this image with linear TV denoising. The next animations show the results obtained , using the fidelity weight λ=1. As can be seen, even with the fidelity term, this model blurs the edges.tvld_cam.gif
  • Now let’s denoise this image with Nonlinear TV denoising. The next animations show the results obtained , using the fidelity weight λ=0.01 and λ=1 respectively.tvn_cam_0.01.gif
    tvn_cam_1

 

Image Inpainting

Inpainting is the process of restoring damaged or missing parts of an image. Suppose we have a binary mask D that specifies the location of the damaged pixels in the input image f as shown below:

f15.png

The following theory is going to be used for TV inpainting.

f16.png

Damaged image  

tampered_cmask1_Cameraman256

cam_in0

Damaged image  

tampered_cmask2_Cameraman256cam_in1

Damaged image  

tampered_text_Cameraman256cam_in2cam_in2

Damaged image
tampered_lena2

lena_in.gif

tv_inpaint_gd_tampered_lena2.png

Advertisements

4 thoughts on “Some more Variational Image Processing: Diffusion, TV Denoising, TV Image Inpainting in Python

  1. Pingback: Sandipan Dey: Some more Variational Image Processing: Diffusion,TV denoising, Image Inpainting in Python | Adrian Tudor Web Designer and Programmer

  2. Pingback: Sandipan Dey: Some more Variational Image Processing: Diffusion, TV Denoising, TV Image Inpainting in Python | Adrian Tudor Web Designer and Programmer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s