Adobe has recently showcased new AI tools for video editing. These tools are designed to make editing faster in Premiere Pro, a popular software from Adobe.
The latest features will be recognizable to users of Adobe’s Photoshop. These features use Adobe Firefly technology to add capabilities like Generative Fill, Generative Expand, and Remove to videos.
The new AI features in Premiere aim to help video editors quickly extend shots or add and remove objects in a scene. These tools will be supported by a new video model that is part of Adobe’s Firefly AI suite. This suite already has tools for Images, Vectors, Design, and Text Effects. The main new tools will be released later this year.
Tools coming out this year:
Generative Extend: This tool uses AI to add more frames, making video clips longer. Adobe states that this advanced technology addresses a frequent issue that professional editors face daily. It allows them to generate additional content to refine edits, extend a shot slightly longer, or improve a scene transition.
Object Addition & Removal: This feature lets users pick and track objects to replace them. It enables the removal of unwanted items, changes to an actor’s clothes, or quick additions like a painting on a wall or flowers on a desk.
Text to Video: This tool generates new video clips in Premiere Pro using text prompts or reference images. These clips can help with creating storyboards or adding B-roll to enhance live action footage.
However, Adobe also gave a sneak peek at something that might be developed later. This new approach is different and could be more controversial compared to Adobe’s work with still images. The company is considering integrating generative AI technologies from OpenAI, Pika Labs, and Runway into Premiere Pro.
Adobe has tried to set itself apart from other generative AI tool developers by promoting Firefly as a more ‘ethical’ and commercially secure AI model. This is because it was trained using public domain assets and licensed images from Adobe Stock, instead of using material collected from the web without permission.
Last week, controversy arose when it was revealed that Firefly’s training data included images from Adobe Stock created by the AI model Midjourney. Now, with video, Adobe appears to be following a path similar to what companies like Canva have pursued, by permitting the use of different AI models within its software.
Adobe says in its press release: “Creative Cloud has always had a rich partner and plugin ecosystem, and this evolution expands Premiere Pro as the most flexible, extensible professional video tool that fits any workflow. Adobe customers want choice and endless possibilities as they create and edit the next generation of entertainment and media.”
Adobe has stated that these are preliminary investigations, and there is no set date for when these integrations might be released to the public. The video demonstration illustrates how, in the future, video editors could use AI video generation models such as OpenAI Sora and Runway in Premiere Pro. This would allow them to create B-roll for their projects. It also demonstrates how Pika Labs could work with the Generative Extend tool to extend the length of a shot by a few seconds.
Interactive fade handles will let editors quickly make custom audio transitions by moving clip handles to set audio fades. The Essential Sound badge automatically identifies audio clips as dialogue, music, sound effects, or ambience and includes a new icon for easy access to the appropriate controls with one click.
Effect badges are visual cues that help identify which clips have effects, allow for quick additions, and open effect parameters directly from the sequence. Additionally, as the track height changes, redesigned waveforms adjust their size intelligently on clips. An AI-powered Enhance Speech tool, which removes unwanted noise and improves poorly recorded dialogue, has been available since February.
What we think?
I think Adobe Premiere Pro’s new AI tools will really change how we edit videos. The AI can add more to a scene and even take things out, which sounds super helpful.
Making videos longer or adding cool details with just a few clicks could save a lot of time. But, mixing AI from different companies might cause some issues or debates about how right it is to use these technologies.
