Adobe brings Firefly generative AI to video editing

Please vote:

Forward-looking: Ever since generative AI-based image creation took off, developers have played with the idea of using AI to generate videos.

While AI can’t create coherent videos from scratch yet, Adobe is adding AI functions to its video editing tools to significantly change the production process.

Adobe is bringing its Firefly generative AI tools to Creative Cloud apps like Premiere Pro

Firefly adds multiple features that could help less experienced users create professional-looking videos.

Firefly can use text prompts to automatically edit a video based on what the AI sees in different shots.

It can combine the feature with audio transcription (which it can use to automatically apply subtitles) to edit scenes according to spoken dialogue.

Adobe demonstrates AI-based editing with a clip showing a woman describing rock climbing preparation.

The original clip is an unbroken shot of her talking in front of the camera, but Firefly picks up her mentions of putting shoes on and applying chalk and edits close shots of those actions on top of the corresponding dialogue.

2023 04 17 image 19

Additionally, Firefly can use text prompts to change color grading.

The demonstration shows phrases like “cool morning” and “golden hour” changing one scene to resemble two different times of the day.

Another text prompt brightens a subject’s face.

Users can also add logos, graphics, fonts, or title cards with text prompts.

Text and logos can be stylized and animated. Furthermore, Firefly can build royalty-free custom music and sounds to add to videos.

Perhaps most strikingly, Firefly can read a script to generate storyboards or edit B-rolls.

Adobe’s video shows the tool creating simple drawn and CG visualizations based solely on the script’s text. Producers can re-arrange the text and quickly see the visual changes.

Adobe first revealed Firefly in late March as an AI image generator attached to apps like Adobe Express, Experience Manager, Photoshop, and Illustrator.

The tool will eventually come to the Creative Cloud, Document Cloud, Experience Cloud, and Adobe Express workflows.

The company is taking multiple steps to avoid the controversy surrounding image generators like Stable Diffusion or Midjourney.

Adobe is currently only training Firefly on Adobe Stock Photos, Public Domain content, and open-license material.

Eventually, the company will give contributors an option to opt out of training its AI and will reveal a system to compensate stock photo makers.




Get The Latest Updates

Subscribe To Our Weekly Newsletter

No spam, notifications only about new products, updates.
On Key

Related Posts

Windows chief sees AI bridging the cloud, PCs

Windows chief sees AI bridging the cloud, PCs

Microsoft has show AI living in the cloud. AMD, Intel, and Qualcomm want AI to live on the PC, powered by their own processors. Does that pose a potential conflict? Apparently not. At AMD’s “Advancing AI” presentation, where the company launched the Ryzen 8040 family of AI-enhanced mobile

Read More »