In the past months, we’ve seen AI meant for generative purposes skyrocket in popularity. Be it texts, images, or even songs, AI has proven itself capable of generating pieces that we could mistake as created by a human. There is one art form yet unaffected directly by AIs but whose individual parts are. An AI today can write a film script as well as generate individual image frames, music, and even voice the dialogues. It is only a matter of time before AI can do all of those things together and produce a full film. In fact, we already got a glimpse at what an AI-generated film would look like with The Frost.
'The Frost' updated with @runwayml. I know we've all seen these before, but this is truly something from nothing. Imagine feeding an animatic brief to an AI of the future and getting back a film! pic.twitter.com/0tsa26eZmq
— Stephen Loggins Parker (@Stephen_Parker) June 10, 2023
AI and film production
While other AI-made film shorts have been made, The Frost was the first one to really go viral. Brought to us by Waymark, a company specializing in AI media, and Latent Cinema, the 12-minute film presents a truly uncanny feeling. The script for the short film was written by Josh Rubin, an executive producer at the company who also directed the film. Besides his input, other humans were involved in editing, production (including AI-generation artists), and animation. This is not a 100% AI film — after all, we’re not there yet — but considering how many people are usually involved in traditional film production, this is still a very remarkable achievement.
The process of creating The Frost involved a delicate dance between human creativity and AI assistance. DALL-E 2 was used to generate the individual frames and D-ID, another AI tool, was used to add movement and animation. “With 15 minutes of trial and error, you get that shot you wanted that fits perfectly into a sequence,” said Rubin in an interview. The process of filming a scene with actors repetitively until you get the perfect shot and clicking generate the same or slightly altered prompt is different but yields a result this film proved similar.
Just wrapped "The Frost: Part One", an AI-generated film! Take a peek at our BTS piece 'From Text to Image' which explores the story of how we came to craft a narrative film using only DALL·E 2 AI-generated imagery. It was an incredible journey… 🧵 pic.twitter.com/01e5qKHgCF
— Josh Rubin (@bigkickcreative) June 22, 2023
The Frost serves as a precursor to the potential of AI in the filmmaking industry. It highlights the evolving collaboration between human creativity and AI assistance, pushing the boundaries of what is possible. Although we may not have an AI-generated full-length feature film I personally dream of just yet, this movie ignites my imagination and anticipation for the future of AI in the world of cinema. And hopefully, all of us will one day be able to generate films from prompts just like we can do now for individual images.
YouTube: The Frost Part One (AI-generated film)
Photo credit: The feature image is symbolic and has been done by Christopher Isak with Midjourney for TechAcute.
Source: Will Douglas Heaven (MIT Technology Review)
