Wants to translate flawless lip movements into different languages

Synchronization, i.e. translating into other languages, is a big challenge in films as well as games. If the spoken texts are translated as true as possible to the original, the lip movements of the visible characters often do not match the visible lip movements. The audience notices it negatively.

Better sync with Deepfax

Various approaches have been developed to avoid this problem. Often the translations are specifically modified so that the content does not change much, but the movements match roughly. In games, on the other hand, it is also possible to create your own facial animations for each language, thus allowing more freedom when translating. In the main example Cyberpunk 2077 Was for example A special AI is used.

The British company is moving in the same direction FlawlessIt wants to change lip movements in images using a neural network. The so-called “true synchronization” technology is comparable to this Well known deep onesFor example, faces are exchanged in video recordings. With real synchronization, special attention should be paid to small facial movements so that the instrument does not change.

More interesting: [PLUS] Knowledge of deep learning: How computers perceive the world

Flawless hat Has already released some videosShowing the application in different scenes and allowing the actors to speak multiple languages. The results are interesting, but the scenes do not always seem completely normal. However, over the next few years, the tool will have to be improved so that its use is no longer noticeable. Cutskins filmed later in movies, and of course games, may benefit from technology.

Inflammation: Wire, Golem

[PLUS] Performance code compared to CPU and GPU, Ryzen 7 4700U and Core i7-1165G7



PCGH Plus: For you in this article we have summarized the different article pages with individual tests and performance codes. Topics: Compared to the Raison 7 4700U & Core i7-1165G7, GeForce RTX 3060 and CPU production in Intel. The article comes from PC Games Hardware 03/2021.
Further …

Go to article

Links marked as * Link links. Link links are not advertisements as we are independent in researching and selecting the products provided. We receive a small commission for product sales, which is used to finance some of the website’s free content.