The Volcano Engine 4K restoration film was selected for the Beijing Film Festival, and related technology won the championship at CVPR2024

On April 18, the 14th Beijing International Film Festival officially opened. The Hong Kong films "Plan A" and "The Pledge of Allegiance" jointly restored by the China Film Archive, Douyin and Volcano Engine were selected into the "Tribute and Restoration" section and will be screened on April 18th and 19th.

4K version of "Plan A" and "Pact of Promise" movie posters

"Plan A" and "Pact of Promise" were released in 1983 and 1978 respectively. It has been more than forty years ago. In August 2023, the China Film Archive, Douyin and Volcano Engine jointly launched the "Classic Hong Kong Movie Restoration Project" and announced the restoration of 100 Hong Kong movies. The two films screened this time come from the above-mentioned projects. The original filming negatives of the films were scanned in 4K format in Hong Kong. The entire restoration process included physical and digital restoration by the China Film Archive, AI technology restoration by Volcano Engine, and cooperation between the two parties. Art restoration and other aspects.

Volcano Engine is a cloud service platform owned by ByteDance. It has worked with Xigua Video and Douyin to complete 4K restoration of 100 domestic classic animations and Beyond classic concerts.

In the "Classic Hong Kong Movie Restoration Project", Volcano Engine applied AIGC technology to old film restoration scenes for the first time, using the powerful generation capabilities of large visual models and rich prior knowledge to achieve restoration effects with higher clarity. , color, fluency and defect elimination, word count: 78.

On the one hand, the large model generation ability is strong. With the help of Stable Diffusion large model generation ability, the repair effect is better than the previous algorithm, and the picture details are more natural; on the other hand, the large model has a very strong information structure. After learning massive data pairs, a large amount of knowledge can be extracted and "stored" in model parameters. Its generation space is richer and can cope with more repair scenarios. Wang Yue, director of video cloud technology architecture of Volcano Engine, introduced.

Currently, the Volcano Engine has made a series of algorithm and engineering optimizations to address issues such as the tendency of large visual models to generate artifacts or false textures in certain scenes, as well as the stability of multi-frame video scenes. It improves the processing efficiency and takes into account the computing power cost in large model scenarios.

According to the in-depth repair work, the Volcano Engine Multimedia Laboratory has carried out a one-step refinement of the large visual model by introducing adaptive feature weighting modules, a combination of unsupervised learning and supervised learning, and text semantic priors. Tuning. It has achieved good results in several aspects such as reconstruction of extremely low-quality portraits, color adjustment of faded film, and generation of missing strokes in text.

Before repair vs after repair

##

Comparison before and after color adjustment of faded film

In order to maintain the "renovated as old" texture of the original film, the technical team not only improved the smoothness of damaged image textures, night scene noise removal and other effects, but also An adaptive texture enhancement scheme was specially developed to add grain that simulates film noise.

At the same time, solving various practical problems in the restoration of old films also provides value to the academic research of the Volcano Engine Multimedia Laboratory. In the top computer vision conference CVPR 2024, the Volcano Engine Multimedia Laboratory won the championship in the NTIRE ( New Trends in Image Restoration and Enhancement) track. Its winning plan was based on The main problem solved by prompt learning (Prompt Learning) image enhancement recovery is to repair unpredictable complex damage (unknown complex degradation) similar to a large number of old movies.

"We actually do the restoration of old films every day." Wang Yue said that common image quality problems such as low definition, low fluency, color distortion, and defects in old films have been restored in It is also widely found in Douyin’s massive submissions. By serving Douyin, Huoshan Engine has formed a relatively complete set of intelligent processing technology solutions for the restoration of old films.

Public information shows that the film restoration process is complicated and trivial, and it takes an experienced restorer one month to complete a film. In recent years, through deep learning and other AI technologies, it has become possible to significantly improve the quality and efficiency of old film restoration.

Ma Kede, assistant professor of the Department of Computer Science at City University of Hong Kong, said that the current rapid development of artificial intelligence technology has shown great potential. In the future, not only the problem of out-of-synchronization of audio and video in some old movies will be solved more effectively through AIGC The solution is even expected to allow AI to learn to automatically judge the repair effect.

Old Hong Kong films carry the youthful memories of many viewers. Jackie Chan’s superb acting skills and thrilling action scenes in "Plan A", as well as Xu Guanwen's humorous performance in "The Pledge of Allegiance" have brought audiences Countless surprises are coming, and with the improvement of restoration technology, these classics from the past will be revived.

During the pre-screening activities of "Plan A", senior film producer Sun Rong, who was the vice president of Jackie Chan China, shared with fans on the spot that Jackie Chan previously thought that the light in the tunnel in "Eagle Project" was not bright enough. , the "Classic Hong Kong Film Restoration Project" team made adjustments based on the opinions and made up for the regrets of his creation back then.

Sun Rong, founder of Dinosaur Pictures, former vice president of Jackie Chan China and vice president of Sparkle Films, shared before the screening

Currently, the audience is on Douyin , Xigua Video, you can watch movies that have been restored by the "Classic Hong Kong Movie Restoration Project" online, and 4K restored versions of other movies will also be launched online one after another.

The above is the detailed content of The Volcano Engine 4K restoration film was selected for the Beijing Film Festival, and related technology won the championship at CVPR2024. For more information, please follow other related articles on the PHP Chinese website!