Google makes Interactions API the default interface for Gemini models and agents

AI General
1

Google Makes Interactions API the Default for Gemini Models and Agents

Google DeepMind has officially designated the Interactions API as the standard interface for all Gemini models and agents. This change replaces the older, text-only API and aims to simplify how developers build multimodal, real-time applications.

The new default API supports text, images, audio, and video input and output in a single, unified session. It is designed to handle long-running, stateful conversations where the model can “see” and “hear” alongside the user.

“This is the new standard for building with Gemini. It’s how all our models will be accessed going forward.”

Why This Matters for Developers

The shift to the Interactions API consolidates multiple fragmented endpoints into one. Developers no longer need to juggle separate APIs for vision, audio, or streaming.

Key Benefits at a Glance

  • Unified multimodal input/output: Send and receive any combination of text, image, audio, or video in a single API call.
  • Real-time streaming: The API supports server-sent events for low-latency, incremental responses.
  • Stateful sessions: Maintain context across multiple turns without manually stitching together history.
  • Agent-ready: Built-in support for tool use, function calling, and multi-step reasoning without extra plumbing.

How the Interactions API Works

The API models interactions as a series of turns between user and model. Each turn can contain multiple parts (e.g., text + image). The model processes all modalities together, then produces a structured response.

Example Flow

  1. User sends a turn with a question and an attached image.
  2. AI processes both and returns a text answer plus a generated diagram.
  3. The session continues, keeping the entire history in context.

This design eliminates the need for developers to manually handle file uploads, temporary storage, or separate audio transcription pipelines.

What Existing Apps Must Do

Applications built on the older Gemini API endpoints need to migrate to the Interactions API. Google DeepMind provides a migration guide and backward compatibility wrappers for a transition period.

The main changes include:

  • Endpoint URL updates from /v1/models/gemini-pro:generateContent to /v1/interactions
  • Request format now requires a session_id for stateful conversations
  • Response structure uses a new “InteractionMessage” object instead of raw content parts

Developers should test their apps now. The old endpoints will remain operational for six months, but new features and performance optimizations will only ship on the Interactions API.

Deeper Technical Details

The Interactions API is built on Google’s Pathways architecture, which allows the model to handle concurrent streams of data. This enables features like real-time voice chat where the model can interrupt or ask clarifying questions.

Supported Modalities

  • Text: plain or markdown
  • Image: JPEG, PNG, WebP (up to 20MB per image)
  • Audio: WAV, FLAC, MP3, OGG (up to 16kHz sample rate)
  • Video: MP4, WebM, MOV (up to 60 seconds, 30fps)

The API automatically converts all inputs into a common representation inside the model, so developers do not need to preprocess files.

Impact on Agents and Tool Use

Google DeepMind is positioning the Interactions API as the foundation for building autonomous agents. The API natively supports:

  • Function calling: Define tools as JSON schemas; the model can decide when to invoke them.
  • Sub‑agents: Launch child interactions that run in parallel, then merge results.
  • Code execution: The model can write and run Python code inside a sandboxed environment.

This means a single Interaction session can involve the model using a calculator tool, searching a database, and generating a chart — all while maintaining a coherent conversation flow.

Migration Checklist for Teams

  • Update SDK version to the latest Gemini client library (v0.9+).
  • Replace generateContent calls with interact or startInteraction.
  • Add session management logic to create and reuse session_id values.
  • Retire any custom multimodal workarounds (e.g., separate audio‑to‑text pipelines).
  • Test latency under different modality combinations; the new API may change performance characteristics.

What This Means for the AI Development Landscape

By making the Interactions API the default, Google is signaling that multimodal, conversational AI is the future — not just for chatbots but for any application that processes diverse data types.

Competing platforms, including OpenAI and Anthropic, already offer multimodal capabilities, but Google’s move integrates them at the API design level rather than as bolted‑on features. This could simplify agent development significantly.

The industry is converging on a single, unified API pattern. Developers who adopt the Interactions API now will be ahead of the curve when other providers follow suit.

Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.

What are your thoughts on this? I’d love to hear about your own experiences in the comments below.