Some Variational Image Processing: Poisson Image Editing and its applications in Python

Poisson Image Editing

The goal of Poisson image editing is to perform seamless blending of an object or a texture from a source image (captured by a mask image) to a target image. We want to create a photomontage by pasting an image region onto a new background using Poisson image editing. This idea is from the P´erez et al’s SIGGRAPH 2003 paper Poisson Image Editing.

The following figures describe the basic theory. As can be seen, the problem is first expressed in the continuous domain as a constrained variational optimization problem (Euler-Lagrange equation is used to find a solution) and then can be solved using a discrete Poisson solver.

f7.png

 

f8.png

 

As described in the paper and also in this assignment from this MIT course on Computational Photography, the main task of Poisson image editing is to solve a huge linear system Ax = b (where I is the new unknown image and S and T are the known images).

 

Seamless Cloning

The following images are taken from an assignment  from the same MIT course, where the Poisson image editing had to be used to blend the source inside the mask inside the target image. The next few figures show the result obtained.

Source Imagebear

Mask Imagemask

Target Imagewaterpool

Output Gray-Scale Image with Poisson Image Editing
pe_waterpool

The next animation shows the solution gray-scale images obtained at different iterations using Conjugate Gradient method when solving the linear system of equations.

peditmit1

Output Color Image with Poisson Image Editingpe_waterpool_color1

The next animation shows the solution color images obtained at different iterations using Conjugate Gradient method to solve the linear system of equations, applying the discrete Poisson solver on each channel.

pbear.gif

 

 

The following images are taken from this UNC course on computational photography. Again, the same technique is used to blend the octopus from the source image to the target image.

Source Imagesource

Mask Imagemask

Target Imagetarget

Output Imagepe_target_color

The next animation shows the solution color images obtained at different iterations using Conjugate Gradient method to solve the linear system of equations, applying the discrete Poisson solver on each channel.

poct.gif

 

Again, Poisson gradient domain seamless cloning was used to blend the penguins’ image  inside the following target image with appropriate mask.

Source Image
peng1

Target Image                                                                                                            trekking

Output Imagepe_trekking.jpg

The next animation again shows the solution color images obtained at different iterations using Conjugate Gradient method to solve the linear system of equations, applying the discrete Poisson solver on each channel.

pe-trekk

 

The next figures show how a source bird image is blended into the target cloud image with seamless cloning.

Source Image
bird1

Target Imagecloud

Output gray-scale imagepe_cloud

The next animation shows the solution gray-scale images obtained at the first few iterations using Conjugate Gradient method when solving the linear system of equations.

pedit2

 

Finally, the next figures show how the idol of the Goddess Durga is blended into the target image of the city of kolkata with seamless cloning. As can be seen, since the source mask had its own texture and there is a lots of variations in the background texture, the seamless cloning does not work that well.

Source Image
madurga

Target Image
kol

Output Image
pe_kol_color

The next animation shows the solution gray-scale images obtained at the first few iterations using Conjugate Gradient method when solving the linear system of equations.

pedit

The next animation again shows the solution color images obtained at different iterations using Conjugate Gradient method while solving the linear system of equations, applying the discrete Poisson solver on each channel.

pe-madurga

 

Feature Cloning: Inserting objects

The next figures are taken from the same paper, here the eyes, nose and lips from the  source face image is going to be inserted into the target monalisa face.

  Source image          Target image         Output image
face mona pe_mona

The next animation again shows the solution color images obtained at different iterations using Conjugate Gradient method while solving the linear system of equations, applying the discrete Poisson solver on each channel.

pe-mona

 

Texture Swapping: Feature exchange with Seamless Cloning

As discussed in the paper, seamless cloning allows the user to replace easily certain features of one object by alternative features. The next figure shows how the texture of the other fruit was transferred to the orange, the images being taken from the same paper.

Source Image        Target Image        Mask Image
sfruit   dfruit    fmask

Output Image
pe_dfruit

The next animation again shows the solution color images obtained at different iterations using Conjugate Gradient method while solving the linear system of equations, applying the discrete Poisson solver on each channel.

pfruit

 

Gradient Mixing: Inserting objects with holes

Again, the next figures are taken from the same paper, this time the source objects contain holes. As can be seen from the following results, the seamless cloning does not work well in this case for inserting the object with holes into the target, the target texture is lost inside the mask after blending.

Source Image                                            Target Image
srch            dsth

Output Image with Poisson Seamless Cloning
pe_tran1

The next animation again shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for seamless cloning, applying the discrete Poisson solver on each channel.

pe-tran1

Using the mixing gradients method the blending result obtained is far better, as shown below, it preserves the target texture.

Output Image with Poisson Mixing Gradients
pe_tran

The next animation again shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for mixing gradients, applying the discrete Poisson solver on each channel.

pe-tran

 

Mixing Gradients: Inserting transparent objects

The next example shows the insertion of a rainbow into a target image, the images are taken from the paper again. As can be seen, the seamless cloning wrongly places the rainbow in front of the coconut tree in the target image. Using gradient mixing, the stronger gradient is used as the image gradient and this solves the issue.

 

    Source Image                              Target Image                         Mask Imagerainbow    sky1      rmask

Output Image with Seamless Cloning
pe_sky
Output Image with Mixing Gradients
pe_rainbow

The next animation again shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for mixing gradients, applying the discrete Poisson solver on each channel.

pe-rainbow

 

The next few figures show the results obtained using mixing gradients on another set of images, the seamless cloning does not work well in this case, but mixing gradient works just fine.

 

Source Image
liberty1

Target Image
vic

Output Image with mixing gradients
pe_vic

The next animation again shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for mixing gradients, applying the discrete Poisson solver on each channel.

pe-vic

Texture Flattening: Creating Flat-texture Cartoon-like images

As illustrated in the paper, by retaining only the gradients at edge locations, before integrating with the Poisson solver, one washes out the texture of the selected region, giving its contents a flat aspect.

The following figures show the output cartoon-like image obtained using texture flattening, using the canny-edge detector to generate the  mask.

Source Image
pe_face1

Mask Image created with Canny edge detector
mface1

Output cartoon obtained with texture flattening from the source with the mask
pe_cart1

The next animation shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for texture flattening, applying the discrete Poisson solver on each channel.

pe-cart1

 

Again, the next figures show the output cartoon-like image obtained using texture flattening, using the canny-edge detector to generate the  mask on my image.

Source Image
me2

Output image obtained with texture flattening
pe-cart-me

The next animation again shows the solution color images obtained at the first few iterations using Conjugate Gradient method while solving the linear system of equations for texture flattening, applying the discrete Poisson solver on each channel.

pe-cart-me

 

Advertisements

4 thoughts on “Some Variational Image Processing: Poisson Image Editing and its applications in Python

  1. Pingback: Sandipan Dey: Some Variational Image Processing: Possion Image Editing and its applications in Python | Adrian Tudor Web Designer and Programmer

  2. Pingback: Sandipan Dey: Some Variational Image Processing: Poisson Image Editing and its applications in Python | Adrian Tudor Web Designer and Programmer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s