Adobe Research engineers have taken a step forward in improving video quality with artificial intelligence. That’s because they’ve just introduced VideoGigaGAN, a revolutionary generative AI model that can turn blurry videos into crisp, detailed footage, with up to 8x the resolution of the original.
This major innovation combines the power of generative adversarial networks (GANs) and new techniques to preserve temporal coherence while avoiding the usual artifacts of super-resolution video. Although still in the experimental stage, VideoGigaGAN opens up promising prospects for video professionals.
Unique technology
VideoGigaGAN is based on GigaGAN, a large-scale super-resolution image model trained on billions of frames. By combining the power of adversarial generative networks with new techniques to preserve temporal consistency, VideoGigaGAN is able to generate ultra-detailed video while avoiding common artifacts such as flickering or distortion. Unlike previous approaches to super-resolution video, which often produce blurry results, VideoGigaGAN achieves the feat of reconciling sharpness and consistency.
The ingenuity of VideoGigaGAN lies in its unique architecture. The model incorporates temporal attention levels into the decode blocks to improve temporal coherence and a stream-driven optical propagation module. An anti-aliasing block is used in the encoder to remove aliasing artifacts. Finally, a hop connection allows high-frequency features to be transferred directly to the decoder layers, thus compensating for the loss of detail. With these innovations, VideoGigaGAN pushes the boundaries of super-resolution video.
What are the uses of VideoGigaGAN?
Although, at this stage, VideoGigaGAN is still just a research project, its potential uses are numerous. For video professionals, this technology could significantly improve the quality of old footage or footage shot in harsh conditions. Integrated with editing software such as Adobe Premiere Pro or After Effects, it would offer a quick and effective solution for “saving” low-resolution shots. VideoGigaGAN could also be used to restore old footage, giving it a second life in high definition.
Beyond the professional world, VideoGigaGAN could also find its place among the general public. In fact, who hasn’t dreamed of being able to improve the quality of their old family videos and other travel memories? With the democratization of social networks, coupled with the craze for artificial intelligence, the demand for such effective video editing tools continues to grow.
And if Adobe decides in the not-too-distant future to integrate VideoGigaGAN into its consumer applications such as Premiere Elements or Photoshop, there is no doubt that this technology will be a great success. And while it’s important to keep in mind that the use of AI to manipulate visual reality and retouch people raises ethical questions that will need to be considered, other players like NVIDIA, Microsoft, and Blackmagic Design are also working on AI video super-resolution technologies, suggesting more competition in this space.