Best Generative AI Method for Image Editing

Hey, this is Denis. Today, I am going to share with you a recently developed Generative AI method for image editing. What excites me is how it can edit images. Instead of replacing objects based on text prompts, it can modify existing objects and move their parts. Although it currently exists only in the form of a scientific paper, I expect this or a similar method to be integrated into many platforms in the future. The code for this paper is expected to be released in June 2023.

Before we start, let me introduce you to our friendly newsletter: Kuration. Explore fresh content across tech, art, and more, curated to ignite curiosity and inspire creativity. From the latest tech breakthroughs and awe-inspiring art to thought-provoking articles and must-see videos.

You can check it out and subscribe here: https://subscribe.kuration.email/

Most modern generative AI methods for image editing, such as Stable Diffusion and its modifications, are based on the Diffusion AI model. We discussed them here and here.

The recently released method "Drag your GAN" belongs to another class of methods — Generative Adversarial Networks. This class is interesting on its own, but I will not cover it today. Instead, let us look at the new image editing model.

Drag your GAN allows you to drag any point in an image in any direction. Here is an example:

1. You want to edit an image of a cat.

2. You want its nose to be on the right side of the image.

3. You just draw an arrow from the existing nose location to the desired one.

4. The model realistically moves the nose in this direction.

How Drag your GAN method works

Let's discuss how it works. In Generative Adversarial Networks, a model uses noise as an input and generates an image from it. By noise, I mean a set of random numbers. You can change this set of random numbers by randomly generating a new set, which will cause the model to change the output image.

So let's return to our cat. If you want to move its nose into a different position, your goal is to find a way to change the input set of random numbers. The real contribution of this method is that it has found a way to do it well. The Drag Your GAN method works with both generated and real images.

My take on Drag your GAN method

1. I see that this opens up a new way of editing images inside photo editors like Photoshop. Instead of just "replace car with a dog," you can realistically move objects in an image.

2. It has other interesting applications. If your child can't sit still during a photo shoot, no problem anymore. Let them move and edit later.

That's all for today. Let me know if you find these posts about methods interesting and useful. Should I do more or less of it? Vote in the poll below.

Cheers,
Denis