Footage shot of New York City was colorized and upscaled using artificial intelligence (AI) over 100 years after it had been taken, and the results are astounding.
In 1911, Swedish manufacturing company Svenska Biografteatern visited the United States and shot extensive footage of the roads. Over a century later, still in mint condition, it had been cut by YouTuber Guy Jones, and slowed down to a more natural rate. The outcome, A Trip Through new york , can be seen below.
From this footage, another YouTuber (enjoying your job here YouTube, favor this a whole lot to your anti-vaxx and pseudoscience articles ) managed to upscale and colorize it with neural networks. In addition to giving it a little colour, Denis Shiryaev fostered the framerate to 60 frames per second, improved the picture resolution to 4K, and revived sharpness. The outcome is something.
The colour was added with DeOldify, an open-source AI tool available on GitHub for retouching images and videos using Generative Adversarial Networks (GAN). It uses two networks — the critic and the generator. It is far more convincing, although it is not the AI to colorize movies and photos.
“This stems from setting up training of the colorizing version to involve another version — the’critic’ – which essentially is there to’criticize’ the colorizations and teach the’generator’ to create better images,” founder Jason Antic informed Hackernoon.
“Since the’critic’ version is also a neural network, it can pick up on lots of the nuances of what makes something seem’realistic’ that simpler methods just can not.”
There has been somewhat of a trend for”deoldifying” early video footage recently (we are not complaining). Earlier this year a restored version of The Arrival of a Train at La Ciotat Station (1896), shot from the Lumière brothers went viral (you’ll see why below).
A train arriving at a station might not seem that thrilling (it is no Shrek, or its sequel Shrek 2) but the first version astonished audiences back when it was first released. You know of the footage of how people reacted to it when they saw it, from stories. The legend goes that when it was first shown, audiences stampeded from the cinema and panicked.
There is no evidence anyone did really run away from the display, and more recently film historians have stated it’s probably just an urban legend. Maybe they would have done if they had seen the version that was restored below?
There are countless other examples on the market, using footage of anything from Doctor Who into the 1935 UK general election. Take a look at a couple more restored videos below.
Though we’ll acknowledge the upscaling can be nightmare fuel.
Upscaling from 16×16 pixels all the way up 4k with 3 Face Super Resolution models chained together. Genuinely impressed by how this one came out. 🙋♀️ pic.twitter.com/RwUkkZAOPw— Jonathan Fly 👾 (@jonathanfly) February 22, 2020