Building AI Agents with LangChain and Vertex AI

The AI agent landscape is evolving rapidly, but the fundamentals are stabilizing. After building several production agents, LangChain paired with Vertex AI has become my go-to stack for anything beyond simple prompt-and-response.

An AI agent is more than a chatbot. It's a system that can reason about a task, decide which tools to use, execute actions, and iterate until the task is complete. LangChain provides the orchestration framework, while Vertex AI provides the foundation models and infrastructure.

The first decision is model selection. For most agent workloads, Gemini Pro through Vertex AI offers the best balance of capability, speed, and cost. For tasks requiring stronger reasoning, Gemini Ultra is worth the premium. The advantage of Vertex AI over direct API access is enterprise features: VPC Service Controls, audit logging, and data residency guarantees.

Tool design is where agents succeed or fail. Each tool should do one thing well, have a clear description that the LLM can understand, and return structured output. I define tools as Python functions with detailed docstrings, the docstring becomes the tool description the model uses to decide when to invoke it.

Memory is the most underappreciated aspect of agent design. Short-term memory (conversation buffer) is straightforward. Long-term memory, remembering user preferences, past interactions, learned facts, requires a vector store. I use PostgreSQL with pgvector through Cloud SQL, which keeps the infrastructure simple and avoids adding another managed service.

Evaluation is hard but essential. I build evaluation datasets of input-output pairs and run them through the agent on every significant change. LangSmith provides tracing and evaluation infrastructure that makes this practical. Without evaluation, you're shipping vibes, not software.

Error handling in agents requires a different mindset. LLMs are non-deterministic, the same input can produce different outputs. Your agent needs to handle tool failures gracefully, retry with modified strategies, and know when to ask the user for help instead of spinning in circles.

The production architecture I've settled on: a NestJS API that receives user requests, dispatches them to a LangChain agent running in a Cloud Run job (for longer execution times), stores results in PostgreSQL, and streams progress updates back through Server-Sent Events.

One lesson learned the hard way: don't over-engineer the agent. Start with a simple ReAct agent with two or three tools. Add complexity only when you have evidence that the simple approach isn't working. The most reliable agents I've built are the simplest ones.