Generative AI and agent services

• Vertex AI / Gemini Enterprise Agent Platform — Google Cloud’s managed AI platform. Announced at Cloud Next in April 2026 as the evolution of Vertex AI; existing APIs, SDKs, and resources continue to work, and both names appear in docs and invoices. This page says “Vertex AI” for the platform and API surface.
• Gemini API on Vertex AI — the enterprise route to Gemini models (IAM, VPC controls, data governance), as opposed to the Gemini Developer API (API key, Google AI Studio).
• Google Gen AI SDK (google-genai) — the unified Python/JS/Go/Java client that targets either the Developer API or Vertex AI.
• Model Garden — catalog of Google, partner, and open models.
• MaaS (model as a service) — a partner or open model hosted and billed by Google per token.
• Grounding — constraining model answers to external sources (Google Search, your data) and returning citations.
• RAG (retrieval-augmented generation) — retrieving relevant documents and putting them in the prompt (see the Retrieval topic).
• Vector Search — Google’s managed approximate-nearest-neighbour service for embeddings.
• ADK (Agent Development Kit) — Google’s open-source, code-first framework for building agents and multi-agent systems.
• Agent runtime / Agent Engine — managed hosting for agents (sessions, memory, scaling). The product has been renamed more than once; check current docs for the exact name.
• MCP (Model Context Protocol) — open protocol for exposing tools and data to agents.
• A2A (Agent2Agent) — open protocol for agents built by different vendors to discover and call each other.
• Gemini Enterprise — Google’s end-user agent and workplace-AI product (successor to Agentspace) with an agent designer and an agent inbox.
• Context caching — storing a large prompt prefix server-side so repeated calls are cheaper and faster.
• Provisioned throughput — reserved model capacity for predictable latency.

 End users          Gemini Enterprise app         Chat apps, your own UI, Workspace
                              │
 Build agents       ADK (code) · Agent Designer (low-code) · prebuilt agents / Agent Garden
                              │
 Run agents         Managed agent runtime (sessions, memory, tracing)  or  Cloud Run / GKE
                              │
 Connect            MCP servers (tools/data)  ·  A2A (agent-to-agent)  ·  APIs, Cloud Functions
                              │
 Ground / retrieve  Vertex AI Search · RAG Engine · Vector Search · Grounding with Google Search
                              │
 Models             Gemini (Pro / Flash / Flash-Lite…) · Imagen · Veo · embeddings · Gemma
                    · partner models (Claude, Llama, Mistral…) · open models (Model Garden)
                              │
 Infrastructure     GPUs / TPUs · Cloud Run · GKE · IAM · VPC Service Controls · Cloud Logging

Everything below the top is consumed through the Vertex AI API surface in your own project, with your IAM, your region, and your billing.

What is available

• Gemini family — multimodal models (text, image, audio, video, PDFs, code) in tiers: a most capable “Pro” tier, a fast and cheap “Flash” tier, and a smaller “Flash-Lite” tier, plus variants for image generation and live/streaming audio. Model names and versions change quickly: look up the current ID in the Model Garden before hard-coding one, and pin versions in production.
• Generation of other media — Imagen (images), Veo (video), speech and music models.
• Embeddings — Gemini and text-embedding models for retrieval (supports output-dimension control, see Embeddings and vector indexes).
• Gemma — Google’s open-weight models, deployable yourself.
• Partner models (MaaS) — Anthropic Claude, Meta Llama, Mistral, and others served as managed APIs, billed per token to your GCP bill.
• Open models — Qwen, DeepSeek, and others you can deploy to your own endpoint from Model Garden (see Deploying ML and AI workloads).

Calling Gemini from Python

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project="my-proj", location="global")   # or a region like europe-west4

resp = client.models.generate_content(
    model="gemini-2.5-flash",                       # example ID: check Model Garden for current models
    contents="Explain IVF indexes in two sentences.",
    config=types.GenerateContentConfig(
        system_instruction="You are a concise technical tutor.",
        temperature=0.2,
        max_output_tokens=300,
    ),
)
print(resp.text)
print(resp.usage_metadata)        # prompt / candidates / total token counts -> what you are billed for

Authentication uses Application Default Credentials (gcloud auth application-default login locally; the attached service account on Cloud Run/GKE), and the caller needs roles/aiplatform.user.

Features worth knowing

Feature	What it does	Why it matters
Function calling	Model returns structured tool calls your code executes	Foundation of agents
Structured output	Constrain responses to a JSON schema	Reliable parsing
Context caching	Reuse a large cached prefix (documents, system prompt)	Large cost and latency saving for repeated queries
Batch prediction	Asynchronous bulk requests at a discount	Offline evaluation, labelling, summarising corpora
Provisioned throughput	Reserved capacity at a fixed price	Production latency guarantees, avoids shared-quota 429s
Thinking / reasoning budget	Controls internal reasoning tokens	Accuracy vs cost (see Reasoning models)
Grounding with Google Search	Retrieves fresh web results and cites them	Current facts, fewer hallucinations
Live API	Bidirectional streaming for audio/video	Voice agents
Safety settings and Model Armor	Content filters and prompt-injection/data-leak screening	Responsible deployment

Quotas, regions, and the 429 problem

Gemini usage is limited by per-project, per-model, per-region quotas (requests and tokens per minute), with shared “dynamic” capacity by default. Under load you get HTTP 429 RESOURCE_EXHAUSTED. Mitigations: retry with exponential backoff and jitter, use the global endpoint to draw on wider capacity, request quota increases, buy provisioned throughput for critical paths, and use batch mode for non-urgent work. A regional endpoint pins processing for data-residency but may have lower capacity.

Three levels of “give the model my data”, from most managed to most custom:

Service	What it is	Use when
Vertex AI Search (the search/RAG product formerly “Enterprise Search”)	Managed ingestion, chunking, indexing, hybrid retrieval, and answer generation over websites, documents, and structured data	You want search or Q&A quickly, with connectors and little code
Vertex AI RAG Engine	Managed orchestration for RAG: import files, chunk, embed, store in a vector backend, retrieve as a tool for Gemini	You want a programmable RAG pipeline without running every piece
Vector Search	Managed, high-scale ANN index (ScaNN-based) with filtering and hybrid search; you control chunking, embeddings, and logic	You need maximum control, large scale, low latency
Database vector support	pgvector in Cloud SQL / AlloyDB, BigQuery VECTOR_SEARCH, Spanner	Vectors live next to relational data, or the corpus is already there
Grounding with Google Search	The model searches the web and cites results	Public, fresh knowledge

# Using RAG Engine as a retrieval tool for Gemini (schematic)
from vertexai import rag
from google.genai import types

corpus = rag.create_corpus(display_name="handbook")
rag.import_files(corpus.name, ["gs://my-bucket/handbook/"], transformation_config=rag.TransformationConfig(
    chunking_config=rag.ChunkingConfig(chunk_size=512, chunk_overlap=100)))

tool = types.Tool(retrieval=types.Retrieval(vertex_rag_store=types.VertexRagStore(
    rag_resources=[types.VertexRagStoreRagResource(rag_corpus=corpus.name)], similarity_top_k=5)))

resp = client.models.generate_content(model="gemini-2.5-flash", contents="What is our parental leave policy?",
                                      config=types.GenerateContentConfig(tools=[tool]))

The concepts of chunking, hybrid search, re-ranking, and evaluation from the Retrieval topic apply unchanged; these services package them. Check the current API signatures in the docs, since this area changes quickly.

ADK is Google’s open-source framework for building agents and multi-agent systems in Python (also Java, Go, TypeScript). It is optimised for Gemini and Vertex AI but model-agnostic (it can call other models through adapters such as LiteLLM), and agents can be run locally, in Cloud Run, or on the managed runtime.

Core building blocks

• LlmAgent / Agent — model + instructions + tools. The reasoning loop.
• Workflow agents — SequentialAgent, ParallelAgent, LoopAgent: deterministic orchestration of sub-agents (pipeline, fan-out, retry loops).
• Tools — plain Python functions (the docstring and type hints become the tool schema), built-ins (Google Search, code execution), MCP toolsets, OpenAPI tools, and other agents used as tools.
• Sessions, state, and memory — session state (key-value, shared among agents in a run), plus long-term memory services.
• Callbacks, guardrails, and evaluation — hooks before/after model and tool calls; built-in eval sets and a dev UI for tracing.

Example: a tool-using agent and a small multi-agent pipeline

from google.adk.agents import Agent, SequentialAgent

def get_exchange_rate(currency: str) -> dict:
    """Return the PLN exchange rate for a currency code such as EUR or USD."""   # docstring = tool description
    rates = {"EUR": 4.27, "USD": 3.95}
    return {"currency": currency, "pln": rates.get(currency.upper())}

researcher = Agent(
    name="researcher",
    model="gemini-2.5-flash",
    instruction="Look up the data the user asks about using your tools. Output only the facts.",
    tools=[get_exchange_rate],
    output_key="facts",                         # result saved to session state under 'facts'
)
writer = Agent(
    name="writer",
    model="gemini-2.5-flash",
    instruction="Write a two-sentence summary for the user from these facts: {facts}",   # state injected into prompt
)
root_agent = SequentialAgent(name="pipeline", sub_agents=[researcher, writer])

Walkthrough: the researcher can call the Python function (ADK builds the function-calling schema from the signature and docstring), and its answer is stored in session state; the writer’s instruction references {facts}, so the second agent sees only the condensed result. This is the orchestrator/worker handoff pattern from Multi-agent architectures, with the wiring done by the framework.

adk web          # local dev UI with chat, traces, and state inspection
adk run my_agent
adk deploy cloud_run --project=my-proj --region=europe-west4 my_agent    # deploy as a Cloud Run service

ADK vs other frameworks

ADK, LangGraph, CrewAI, and the OpenAI Agents SDK all implement “agent = model + prompt + tools + loop”. ADK’s differentiators are tight Vertex AI/Gemini integration, built-in workflow agents and evaluation, native A2A and MCP support, and a deployment path to the managed runtime. LangGraph offers finer-grained graph state control. You can mix them: ADK supports running agents from other frameworks as sub-agents via A2A.

Option	What you get	Trade-offs
Managed agent runtime (Vertex AI Agent Engine; renamed Agent Runtime in some current docs)	Hosting for ADK, LangGraph, LlamaIndex, or custom agents with managed sessions, memory bank, scaling, tracing, and IAM; deploy from Python	Opinionated; fewer knobs; verify current naming and feature set in docs
Cloud Run	Your container, any framework, your own session store (Firestore, Redis, Cloud SQL), scale to zero	You build memory/session plumbing
GKE	Full control, long-running agents, sidecars, GPUs for local models	Cluster operations

# Deploy an ADK agent to the managed runtime (schematic: exact module paths evolve)
import vertexai
from vertexai import agent_engines

vertexai.init(project="my-proj", location="europe-west4", staging_bucket="gs://my-staging")
remote = agent_engines.create(
    agent_engine=agent_engines.AdkApp(agent=root_agent),
    requirements=["google-cloud-aiplatform[agent_engines,adk]"],
    display_name="fx-assistant",
)
session = remote.create_session(user_id="u1")
for event in remote.stream_query(user_id="u1", session_id=session["id"], message="EUR in PLN?"):
    print(event)

Production concerns for agents

• Identity. Give each agent its own service account with least privilege; the agent acts with that identity when calling tools and APIs.
• State. Sessions and memory must survive instance restarts (managed runtime, Firestore, or a database), since serverless instances are ephemeral.
• Cost and loops. Cap turns, token budgets, and concurrency; agents multiply model calls.
• Observability. Cloud Trace and Logging integration (OpenTelemetry) for every model call and tool call; log prompts carefully.
• Security. Model Armor for prompt-injection screening, VPC Service Controls to keep data inside a perimeter, and Secret Manager for credentials. Treat tool output and retrieved text as untrusted (see Safety and failure modes).
• Evaluation. Use ADK eval sets and Vertex AI’s Gen AI evaluation service to regression-test agent trajectories before deployment (see Agent evaluation).

MCP — agent to tools and data

Google supports MCP across its stack: ADK agents can mount MCP servers as toolsets, and Google offers MCP servers and “toolbox” projects for its own services so agents can query databases (BigQuery, AlloyDB, Cloud SQL, Spanner) and manage cloud resources through a standard protocol. You can also host your own MCP server on Cloud Run (remote MCP over HTTP with IAM authentication). See MCP, skills, harnesses, and guidelines in the AI agents topic for the protocol itself.

from google.adk.agents import Agent
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset, StreamableHTTPConnectionParams

agent = Agent(
    name="db_agent",
    model="gemini-2.5-flash",
    instruction="Answer questions about sales using the database tools.",
    tools=[MCPToolset(connection_params=StreamableHTTPConnectionParams(url="https://my-mcp-xyz.a.run.app/mcp"))],
)

A2A — agent to agent

A2A (Agent2Agent) is an open protocol, originated by Google and now under the Linux Foundation, for one agent to discover and call another agent as an opaque peer, even from a different vendor or framework:

1. Each agent publishes an agent card (JSON at a well-known URL) describing its identity, skills, endpoint, and auth.
2. A client agent sends a task to it over HTTP (JSON-RPC, with streaming for long tasks).
3. The remote agent returns messages and artifacts, with task states (working, input-required, completed).

MCP:  agent ──► tool / data source     (agent uses a capability)
A2A:  agent ──► another agent          (agent delegates to a peer that has its own reasoning and tools)

Use MCP for tools, A2A when the other side is itself an autonomous agent owned by another team or vendor. Within one codebase, ADK sub-agents are simpler than A2A.

• Gemini Enterprise (the successor to Agentspace) is a packaged workplace-AI product: employees chat with agents that can search across connected company data (Workspace, Microsoft 365, Salesforce, Jira, and others), run research, and take actions. It includes an Agent Designer (no-code/low-code agent builder), an agent inbox for monitoring long-running agents, a gallery of Google and partner agents, and governance controls for IT.
• Prebuilt Google agents (deep research, data science, code assistance, customer engagement/contact-center agents) can be used as-is or extended.
• NotebookLM Enterprise — source-grounded notebooks and audio overviews over company documents.
• Customer-engagement suites (conversational agents with voice, formerly Dialogflow CX) for contact-center use cases.
• Gemini in the developer tools — Gemini Code Assist, Gemini CLI, Gemini Cloud Assist (ask questions about your own cloud environment), and BigQuery/Looker assistants.

Which layer should you use?

Need an assistant for employees over company data, minimal code?     -> Gemini Enterprise (+ Agent Designer)
Need a custom agent embedded in your product, full control?           -> ADK + Cloud Run or the managed runtime
Need only a model call or two in an app?                              -> Gemini API on Vertex AI directly
Need search / Q&A over documents fast?                                -> Vertex AI Search
Need agents from different teams/vendors to interoperate?             -> A2A (and MCP for tools)

Staying current

Product names, SKUs, and model IDs in this area have changed several times in a year (Agentspace -> Gemini Enterprise; Vertex AI -> Gemini Enterprise Agent Platform; Agent Engine -> Agent Runtime in some docs; successive Gemini versions). When building, treat this page as the map and the official docs and Model Garden as the source of exact names, flags, and prices.