Interview with Simian and Anthony Vagnoni

There’s been much wailing and gnashing of teeth in the advertising production and post industry. The twin threats of growing in-house agency production and AI have caused some to say, “It’s just a cycle that will see a swing back as it always has”. To others who lament that “It was a fun business while it lasted!”. 

Over the past 8 months or so, I’ve been attempting to gauge how our industry is going to shake out in the future. For those who don’t know me, I’m a long-time DGA director and DP.  I founded or co-founded several production and post companies including Click 3X, Sound Lounge, Heard City, ClickFire Media and Bonfire. Recently I worked with a Madrid-based company to develop their virtual cloud-based workstation business. 

Several months ago, I began helping a creative editorial company in researching in very specific terms, where AI can be instrumental in post. This was less about generative AI - creating images and video from a set of prompts - and more about how AI can work in post now. 

Literally every day, I scan the half-dozen new AI apps then check out and try to test those that impact media production.  Here’s what I see. 

There are applications including existing tools that have been AI-infused and dramatically alter how we should approach post. From color-correct to beauty retouching to rotoscoping to editing to audio recording to stock footage selection and manipulation to CGI to design to all manner of VFX to animation - even the way production is managed - everything has changed. Every skill listed here has an AI-fueled solution that require way less craft to execute.

The AI capabilities in post flow directly to production. If AI can up-res shots, create slow motion, clean up faces, add lighting, remove unwanted objects, replace skies, automatically rotoscope anything and replace entire environments etc. then, “We’ll fix it in post” will be replaced by, “We’ll reconstruct it all with AI”. The production cost savings on many shoots could be seismic.

As an industry we have relied on the brilliant original creative inspiration to concept, direct, design and sell the treatment and then an entire host of craft people to carry out the execution. We make some money on the original concept, but we make most on the execution including versioning, client changes etc. AI literally replaces that much of this less expensively, more efficiently, and most important, without the investment in capital, infrastructure and teams that form the backbone of so much of our business. 

So, who owns these AI processes and prompt skills? The Holding Companies are investing hundreds of millions in AI initiatives - including acquisitions - in order to own this. They too are in a race with their clients to make sure they remain relevant. Two-thirds of brands already have in-house agencies who are also well-positioned to invest in these AI systems. 

While other technologies have taken years to filter through the production process, AI is moving so incredibly fast that new uses and applications will come on the market fully developed in months – not years.  Nvidia just announced delivery of their new AI superchip Blackwell whose 208 billion transistors is 30 times faster than their previous most advanced chip.  So, whatever you think of Sora, imagine AI video production 20 or 30 times more advanced. BTW Sora along with ChatGPT5 is set to launch in the market as early as this fall. 

In conclusion, my view is that there is an inevitability that the business of production/post for agencies and brands will shrink significantly. There are still thousands of clients and agencies who will not have near-term access to this new wave and TV commercials will continue to thrive as an industry as more and more streamers become ad supported. But the writing (voice prompt?] is on the wall and entire craft categories and careers will virtually disappear. I wouldn’t want to count on making a living as an extra or a rotoscope artist! 

I do think that as best we can, the production/post industry needs to familiarize themselves with and USE the new tools – not just experiment. This won’t stop the inevitable, but it will postpone the coming storm.

In future postings, I’ll list some of the developments and tools that I think directly impact production and post.

Peter Corbett

While everyone is focused on generative AI’s ability to create incredible video, useful AI can be found in the applications we have in our toolbox right now. 

Firstly, I believe that in-house agency production and in-house brand agencies are not likely to have any significant advantage in the implementation of visual generative AI. Agencies are spending hundreds of millions on AI, but much of this investment relates to targeting, media, and consumer research. When it comes to Runway ML, Midjourney, Pika etc. everyone is generating from the same LLMs. While these agencies may control the work and continue to steer projects in-house, they will have little executional advantage.

The breathtaking speed of progress simply demands that production and post shops need to broadly embrace AI. Four Areas:

1. Production – Live action filming. AI capabilities that are readily accessible and potentially offer huge production efficiencies are embedded in the software we use now. Blackmagic’s Resolve is a good example. Topaz ($299) is another. These programs use AI to up-res up to 8K, interpolate to create slow motion shots without high-speed camera equipment, create actual depth maps - even add lighting effects where none existed. The potential savings and efficiencies once a production company filters these new capabilities to the shoot itself will be significant.

2. Post. AI can not only rotoscope but will also replace environments - even with dynamic camera moves. Color-correct and beauty retouching as well as sky replacements etc.  - which can be a substantial component of a budget - can be achieved at a far lower cost than previously.  Adobe’s new version of Premiere being released this year, will enable the editor to magically extend scenes, eliminate objects within the shot and even add and replace objects with AI - all within the edit timeline. Since Premiere will also integrate with other generative AI platforms like Sora, Runway ML and Pika, a substantial part of creating a film could be in the hands of the editor.

3. Stills to Video: The capability of AI programs like Runway ML to take an existing still image and create motion opens a huge opportunity to build new shots and sequences. Imagine a product shot that would normally require a set up with a full camera crew etc. can can use a still, then literally bring the image to cinematic life - add foliage rustling in a breeze, cloud movement, a dynamic camera move, drops of condensation rolling gently down the side of the product, dynamic lighting effects etc.  All possible from a single still! 

4. Creating new shots and sequences from scratch. Sora is being released in a few months as part of Chat GPT5. Right now, working with Sora is a little like a roulette wheel. You enter the prompts, never sure exactly what you’re going to get. Trying to noodle and finesse can result in frustrating random changes. However, this is where OpenAI’s goal of improving these programs is being focused. Adobe’s Firefly is a good example of such improvement where you can now dictate focal length, f-stop, depth-of-field lighting style etc. 

Since Sora can build spectacular shots and full sequences that are either impossible or extremely expensive, integrating this immeasurable production value into a story is now not only available to the filmmaker, but also to his editor, his client and even the team cranking out an Instagram brand post!  There’s a revealing interview on Sora with Mira Murati Open AI’s CTO in The Wall Street Journal (subscription required). In this interview the question was raised about where the videos used to train the LLM came from. OpenAI has a deal with Shutterstock, but the issue of copyright safety is going to loom large.

I know that as soon as I put out this piece, there’s a high likelihood it will almost immediately be outdated. There is now an entire community of AI filmmakers who are producing all-AI films. Out of this community will emerge directors and editors who inevitably will permeate into the commercial production business. OpenAI opened up Sora to several directors to try to make a short film just using voice prompts to generate the shots. I’d like to share one example from this from Walter Woodman of Shy Kids, “Air Head”.  You can also check out behind-the-scenes here

Finally, to the many of you already experimenting and using AI tools. I would encourage you to comment and share your own experiences and advice.