Adobe Firefly Adds AI Audio Tools and Google’s Gemini Omni Flash
Adobe has expanded its Firefly AI platform with new audio generation tools and integrated Google’s Gemini Omni Flash model. The update adds text-to-audio capabilities and real-time multimodal processing, directly within workflows like Premiere Pro and the Firefly web app.
Users can now generate sound effects, ambient audio, and voiceovers by typing a description. Google’s Gemini Omni Flash allows Firefly to understand and edit video and audio simultaneously, enabling tasks like removing background noise and matching lip movements.
What Adobe Firefly’s AI Audio Tools Can Do
Text-to-Audio Generation creates sound effects and music tracks from simple prompts. For example, typing “rain on a window” produces a matching audio clip that can be dropped into a timeline.
Voiceover Synthesis generates spoken narration with selectable tones and accents. The tool is designed for quick prototyping, not final production, and includes watermarking to indicate AI origin.
Ambient Sound Creation builds layered background audio for scenes. Users control mood, intensity, and duration without needing a recording studio or sound library.
How Google’s Gemini Omni Flash Changes Firefly
Real-Time Video and Audio Understanding is the core addition. Gemini Omni Flash can analyze both visual frames and audio tracks simultaneously, allowing Firefly to edit based on what it “sees and hears.”
Automatic Voice Matching syncs generated dialogue with on-screen mouth movements. The model adjusts timing and pitch to match existing footage, reducing manual lip-syncing work.
Background Noise Removal uses the multimodal model to isolate specific sounds. Users highlight unwanted audio, and the AI removes it while preserving voice and essential effects.
“This is the first time we’ve combined video and audio understanding in a single model for creative tools,” said an Adobe product manager. “Gemini Omni Flash allows Firefly to process context, not just isolated clips.”
What This Means for Video Editors
Faster Prototyping is the main benefit. Editors can generate placeholder audio and test scenes without recording or licensing tracks. Final audio work still requires human oversight.
Tighter Integration between Adobe products reduces context switching. Tools like After Effects and Audition now share AI audio features through the Firefly backend.
Ethical Safeguards include mandatory content credentials. All AI-generated audio carries a “do not train” tag, and Adobe prohibits using the tools to clone copyrighted voices or create deceptive content.
Adobe’s Competitive Position
Google’s Gemini is a strategic partner, not a competitor in this case. Adobe uses the model via an API, while keeping Firefly’s core generation engine proprietary.
OpenAI’s Voice Engine and Meta’s AudioCraft are direct rivals. Adobe differentiates by embedding AI tools directly inside paid creative software, not as standalone apps.
Enterprise licensing allows companies to use generated audio in commercial projects without additional fees, as long as they hold an Adobe Creative Cloud subscription.
Privacy and Data Handling
Generated audio is stored on Adobe’s servers for 60 days and then deleted. Users can opt out of data collection for model training via account settings.
No local processing occurs yet for audio generation. Unlike some Firefly image features, audio tools currently require an internet connection to access the Gemini model.
Watermarking is mandatory for all AI audio output. Adobe’s Content Credentials system embeds metadata that identifies the tool, model, and generation prompt.
Availability and Pricing
Firefly Audio is rolling out now in beta for Creative Cloud subscribers. The feature is included in standard plans, but heavy usage may require a premium add-on.
Gemini Omni Flash integration is available in Premiere Pro version 24.6 and later. Adobe plans to expand multimodal features to other apps by early 2025.
No separate Firefly Audio subscription exists. Users need a Creative Cloud All Apps or video-specific plan to access the tools.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.