Sometimes it becomes very tedious to read subtitles of foreign film, and that is why most movies are dubbed into different languages. But dubbing needs perfection, otherwise, it might ruin the feel of the movie. Most of the time these dubbed dialogues don’t match the actor’s lip movements. To tackle this problem, a company named Flawless has developed an AI-powered algorithm that will replace the actor’s facial performance to match the words in the dialogue.
This tool has a lot of potential for making films more accessible and affordable. However, such artificial intelligence and neural network-powered tools can be used to manipulate stills and videos of public figures to create situations that never actually happened.
Flawless believes that artificial intelligence is capable of creating “perfectly lip synchronized visualizations in various languages.” The company has posted a showreel that is very convincing because it can be seen that the technology does not have any effect on the facial expressions of the actor but somehow the dialogues feel more natural than in the traditional way of dubbing.
Flawless said that TrueSync software uses the same tools responsible for deepfake videos to manipulate and adjust an actor’s face in a film so that the movements of their lips are more closely matched to how they would have been in the scene was originally performed in that respective language.
TrueSync analyzes the actor’s face to detect its movement and renders it using neural networks and smartly does the required tweaks without affecting the entire face. The system was in development for years, and now finally the company has launched it commercially.
In the video, we could see two more clips from the movie ‘Forrest Gump’. The film originally is shot in the English language but in the video, we can see the implementation of TrueSync, dubbing the clips in Spanish and Japanese perfectly.
The technology is progressing. In today’s world, one can produce Hollywood-caliber films just using a smartphone and a laptop. Artificial Intelligence does make this process easier and less time-consuming. The more we research, the more we can produce affordable movies and also save time. We can already see the progress in Hollywood movies like Terminator, Jurassic Park, or Movies from Marvel, where the use of artificial intelligence and computer graphics is so flawless that one can think it is all real. With TrueSync, filmmakers will be able to produce movies in different languages with seamless lip-sync and will also enable them to convey the message of the movie exactly as they were intended originally without losing the authenticity.
The recent Academy Award-winning film Parasite has seen a huge success all over the world. This indicates that people do watch movies in foreign languages. Reading subtitles is considered distracting. Dubbed movies often fail to justify the actor’s performance. The emotions and the feeling that an actor put in his lines are lost during dubbing. To overcome this and keep the original essence of the films, TrueSync might be of some help.
