Agents
Distilled from Google's "Introduction to Agents" whitepaper (Day 1; updated May 2026) — the first of the 5-part Gen AI Intensive. My own paraphrase + takeaways, not the original text.
One-liner: an agent is "LMs in a loop with tools to accomplish an objective." A model alone predicts/creates; an agent reasons, acts, and observes on its own until a goal is met.
See also the AI Engineering Glossary for Agent, RAG, Tool use, Orchestration layer, ReAct, Context engineering, HITL, A2A, Agent Ops.
The 4 core components
| Part | Analogy | Role |
|---|---|---|
| Model | Brain | The reasoning engine (LM). Picks plans, evaluates options, decides |
| Tools | Hands | Connects reasoning to the world — APIs, code, data stores, RAG |
| Orchestration Layer | Nervous system | Runs the loop; manages planning, memory/state, reasoning (CoT, ReAct) |
| Deployment | Body & legs | Hosting, monitoring, logging — turns a prototype into a real service |
Mental model: an agent is context-window curation in a loop — assemble context → prompt model → observe → re-assemble for the next step.
The agentic loop (5 steps = "Think, Act, Observe")
| Step | What happens |
|---|---|
| 1. Get the Mission | A high-level goal arrives (user or trigger) |
| 2. Scan the Scene | Gather context: request, memory, available tools |
| 3. Think It Through | Reason out a plan (often a chain of steps) |
| 4. Take Action | Invoke a tool — call an API, run code, query a DB |
| 5. Observe & Iterate | Add the result to context; loop back to Step 3 until done |
Example: "Where's order #12345?" → find_order() → get_shipping_status() → report "Out for delivery."
Taxonomy — 5 levels of agentic systems
| Level | Name | Capability |
|---|---|---|
| 0 | Core Reasoning | LM alone, no tools — knowledgeable but "blind" to anything post-training |
| 1 | Connected Problem-Solver | Adds tools (search, APIs, RAG) → can reach real-time facts |
| 2 | Strategic Problem-Solver | Context engineering — plans multi-step goals, curates context per step |
| 3 | Collaborative Multi-Agent | A team of specialists; agents treat other agents as tools |
| 4 | Self-Evolving | Identifies its own capability gaps and creates new tools/agents on the fly |
Building the agent — key design choices
- Model selection: don't chase benchmarks — test on your task; balance quality vs cost vs latency. Often route: frontier model for hard planning, cheap/fast model for simple subtasks.
- Tool types:
- Retrieve (grounding): RAG over vector DBs / knowledge graphs; NL2SQL for structured data → reduces hallucination.
- Act (change the world): wrap APIs/code; run code in a sandbox; HITL tools to pause for human confirmation.
- Function calling: OpenAPI specs or MCP give the model a structured contract so it calls tools reliably.
- Instruct with persona: the system prompt is the agent's "constitution" — role, constraints, output schema, tone, when to use tools. A few examples help a lot.
- Memory:
- Short-term = scratchpad of the current conversation (Action/Observation pairs).
- Long-term = persistence across sessions, usually a RAG system over a vector DB.
Multi-agent design patterns
| Pattern | Use it for |
|---|---|
| Coordinator | Dynamic/non-linear tasks — a manager agent routes sub-tasks to specialists, then aggregates |
| Sequential | Linear pipelines — output of one agent feeds the next (assembly line) |
| Iterative Refinement | Quality — a generator + a critic agent loop against a rubric |
| Human-in-the-Loop (HITL) | High-stakes — pause for human approval before a significant/irreversible action |
Agent Ops — testing the unpredictable
Traditional output == expected fails when responses are probabilistic. Agent Ops = DevOps/MLOps adapted for agents:
- Measure what matters: define KPIs like a business A/B test — goal-completion rate, satisfaction, latency, cost/interaction, revenue impact.
- LM-as-Judge: a strong model scores outputs against a rubric, run over a golden dataset. Have a domain expert validate before trusting scores.
- Metrics-driven go/no-go: run the new version against the eval set, compare to production; use A/B rollouts.
- Traces (OpenTelemetry): step-by-step recording of the agent's trajectory — answers "why did it do that?"; for debugging, not dashboards.
- Cherish human feedback: every 👎 / bug report = a new edge case → turn it into a permanent eval case ("close the loop").
Key insight: "comprehensive evaluations often outweigh the initial prompt's influence." Evals > prompt tweaking.
Interoperability & the agentic ecosystem
- A2A (Agent2Agent): open standard for agents to discover + talk to each other. Each publishes an Agent Card (JSON: capabilities, endpoint, credentials). Task-oriented & async. MCP connects agents to tools; A2A connects agents to agents.
- Agents ↔ humans: chatbots, structured JSON UIs, computer use (LM drives a UI), live multimodal (voice/vision) via streaming APIs.
- Agents ↔ money (emerging): AP2 (cryptographically-signed "mandates" = proof of user intent) and x402 (machine micropayments via HTTP 402).
Security — the trust trade-off
More power (tools, autonomy) = more risk (rogue actions, data leakage). Defense-in-depth:
- Deterministic guardrails — hardcoded rules outside the model (e.g. block purchases > $100, require confirmation).
- Reasoning-based defenses — small "guard models" screen plans/inputs for prompt injection & policy violations (e.g. "Gemini as a Judge").
- Agent Identity — a new class of principal (alongside users & service accounts). Each agent gets a verifiable cryptographic ID (e.g. SPIFFE) + least-privilege permissions → contains the blast radius.
- Governance at scale — as agents proliferate ("agent sprawl"), route everything through a central gateway + registry (control plane) for auth, policy, and observability.
How agents self-evolve (frontier)
Agents "age" as the world changes; better to let them adapt. They learn from runtime experience (logs, traces, HITL corrections) and external signals (new docs/policies), then improve via:
- Enhanced context engineering — refine prompts, few-shot examples, retrieved memory.
- Tool optimization/creation — gain, build, or modify tools to close capability gaps.
- Agent Gym — an offline simulation/training environment (not in the live path) for trial-and-error, synthetic data, and red-teaming.
Advanced examples: Google Co-Scientist (research collaborator; supervisor + specialist agents) and AlphaEvolve (evolutionary code generation + automated evaluation — strong where verifying a solution is easier than finding it).
The takeaway
The developer's role shifts from bricklayer (write every logical step) to director (set the scene, pick the cast of tools, supply context, then guide the autonomous actor). The flexibility that makes LMs powerful is also what makes them unreliable — so success comes from engineering rigor: solid tool contracts, error handling, context management, and evaluation.
Open questions
Things to revisit — promote to Open Questions if they stick around:
- When is Level 3 (multi-agent) actually worth it vs a single well-engineered agent? (Day 1 hints at LM reasoning limits; cross-check against Anthropic vs Cognition's debate.)
- ReAct vs Chain-of-Thought — when does each orchestration strategy pay off?