Comixify: Transform video into a comics. They used a 2-stage approach: (a) frame selection and (b) style transfer. The results look pretty cool!

paper: https://t.co/cYmCk5YosP

web demo: https://t.co/QZq41b2TnN pic.twitter.com/FGcPfsN9u2 — hardmaru (@hardmaru) December 12, 2018

According to the researchers, the project uses a neural style algorithm based on Generative Adversarial Networks (GANs). The machine learning technique by using two different networks: a generator and a discriminator. The algorithm is fed tons of information, which the generator uses to create new data instances. The discriminator is then tasked with determining if the instance is from the dataset it was trained on or generated by the other network. The process continues until the generator is producing work so similar to the training set that the discriminator can no longer tell the difference.

Because comics only have so many frames, the system has to determine the best, most representative moments from a video to create the comic. Once the algorithm has picked its frames, it then applies the comic style to them. The end result is a pretty reliably solid reimagining of any given video as a page from a comic book.

A full paper detailing how Comixify works has been published online. If technical jargon isn't your thing, you can skip right to feeding the algorithm video through a web application.