What is deep fake?

What is a deep fake? And what happens behind all these convincing videos and images?

ZARA PIVATO

MEDIAMATIKERIN IN AUSBILDUNG

20th February 2023 | Tags: Technology, Deepfake, AI

With today’s technology, we’re able to create a ton of fake content that can mislead the viewers or seem offensive to an individual. Due to that reason it is important that we’re all able to tell fake and real apart. The following Blog not only covers questions about what deep fake is, what it contains and how it works, but also the history behind it and how you, as well as everyone else, can easily see whether the videos you come across actually happened or are a deep fake.

Table of content

What is deep fake?
Which are the four different types of deep fake?
When did deep fake first appear?
How does swapping faces with deep fake work?
How does deep fake work?
How does facial manipulation with deep fake work?
What are deep fakes for?
What are the pros and cons about deep fake?
Is it a crime to create a deep fake?
How do you spot a deep fake video?

Deep fake is an artificial intelligence that’s also known with the name deep learning. The term describes the technology behind it, as well as the result you get after creating such a video. They mainly get made to create convincing images and videos by, for example, swapping people’s faces with each other to make it seem as real as possible and fool people. It’s also very common to use deep fake to put words into people’s mouths and let everyone think they actually said the spoken words.

Which are the four different types of deep fake?

1. Face swapping

When using this type of deep fake, what happens is, the face of person A gets placed on the body of person B.

2. Facial manipulation

Facial manipulation is a more complicated process, because it involves a lot of tracking. The facial expressions of person A get placed on person B, while using this technique. It can also be used in live reenactments if prepared correctly.

3. Audio

It is even possible to let your chosen sentences get spoken by a celebrity’s voice, or even make yourself sound like one in a call. With specific programs anyone is able to make it happen.

4. Creating a non-existing person

With Deep Fake you can create non existing people, simply by mixing two peoples features with each other, with the help of AI. This often gets used for marketing purposes.

When did deep fake first appear?

Back in 1997 deep fake started with a video rewriting program created by Christoph Bregler, Malcolm Slaney and Michele Covell. It was able to change footage into new videos, where they mouth words they didn’t actually say. Deep fake was getting more known later in 2017.

How does swapping gaces with deep fake work?

To make it easier to understand what happens during face swapping, take a look at the picture. In this example the face of person A got placed onto Person B’s body, making it look like they belong together.

How does deep fake work?

For starters, it’s important to have high-quality footage of each person. When possible, aim for at least 300 photos or videos per person – the more data you provide, the more accurate the results will be.
All of this footage is then fed into individual encoders. This allows AI to analyze different facial features and structures of each person. Once the process is complete, the images are compressed into what’s known as a Latent Face (see above).
The compressed images are then loaded into a separate decoder, where they’re reconstructed using differences in facial features that were identified in Step 2. For example, if Person A had a happy smiley face but Person B had a sad one, once this information is fed through the decoder, Person A will now look sad as well.
For optimal results and increased realism in the end product video, this same process must be applied to every single frame of footage used.

How does facial manipulation with deep fake work?

Facial manipulation is a bit more complicated, since every movement has to get tracked by the small orange dots. What happens here is that the facial expressions of person A got tracked and placed on Person B. Making it easy to put words in peoples mouths.

What are deep fakes for?

In total, counted in 2020, 85’000 videos have been found on the Internet that are a deep fake. Astonishing 96% of these are pornographic, most of them are videos where the actors face got switched with one of a female celebrity. As Danielle Citron said, a professor of law at Harvard University, deep fake videos are getting weaponised against women. The rest of these countless videos get sorted under categories such as movies, fake news, jokes and bullying.

What are the pros and cons about deep fake?

Even though the biggest part of deep fake videos get used against others, there are also good aspects to this technology.

Pros

Restoring voices

With AI it’s possible to bring back people’s voices that lost them due to an illness, but also let people, who are coping with the loss of someone close, hear them talk some words again.

Museums and galleries

Films

Deep fakes can help improve the synchronization of films that got produced in a foreign language. The technology also gets frequently used when actors pass away and they want to bring them back for a short scene in a film.

Cons

Fake news

People often believe anything they see online, that’s how fake news get spread easily. Especially if the producer of the deep fake looked out for every detail, it gets more dangerous for us. They can put anything into other peoples mouths and if it’s with a celebrity we instantly buy it.

Not appropriate content

The pornographic videos that are then edited with the help of deep fake, get made without the consent of the person’s face they used. Celebrities like Taylor Swift are the most common victims in these situations. But also bullying has taken up a big place in deep fake videos, they get portrayed in a ridiculous way which can be offensive.

Is it a crime to create a deep fake?

It is in fact not illegal to create a deep fake, yet if it violates any of the following topics it can lead up to two years of jail time: copyright, data protection law, making a victim look ridiculous, and sharing private images or videos without the consent of the shown person.

How do you spot a deep fake video?

There are plenty of details to watch out for if you’re not sure whether a video is fake or showing the truth.

Eyes

If you only give the AI pictures where the person has their eyes open, it doesn’t know what they would look like with closed eyes, so it makes up its own version. So if weird blinking is involved, it may be a deep fake. A small detail is also the light reflection on the iris.

Mouth

One of the biggest factors is bad lip synching, it’s the most obvious out of them all in general. But also the teeth may look weird, that also stands in connection with the pictures you give the Algorithm. If your person is mainly smiling without showing their teeth, the AI has to make up its own again.

Skin

The skin tone can start to look patchy and full of small gaps if the person is moving a lot in your targeted end result.
If the person’s face lighting doesn’t match with the one from the background, you can be sure that you’re getting fooled. Generally weird lighting is something to keep an eye on.

Face

As before, when there is much movement involved, the edges around the face can start to flicker weirdly, which would never happen in a regular video.

Hair

Fine detailed hairstyles or strands may also start to flicker, as well as disappear and reappear from time to time.

Conclusion

Don’t believe anything you come across on the Internet, in today’s time it gets easier and easier to fool each other. This can lead up to us not knowing which sources to trust anymore. In the future we can all build our own subjective reality, if that’s a good thing or not is something everyone has to decide for themselves. But most importantly, always keep the factors about how to spot a deep fake, in the back of your head, and you will be safe for the foreseeable future.