Building Your First AI Agent with LangGraph

Building Your First AI Agent with LangGraph

In July 2025, SaaStr founder Jason Lemkin was testing Replit's AI coding agent in public. He told it, in writing, not to change anything without his permission: a code freeze. On the ninth day the agent deleted his production database anyway, more than 1,200 executive records and nearly 1,200 company profiles. When he asked whether the data could be restored, the agent said no. That was wrong too. Replit's own rollback worked, and he got the data back. Replit's CEO called the incident unacceptable, and the company's first fix was structural: development and production databases were separated automatically, so the agent could no longer touch live data by default.

That story is the whole reason this part exists. The agent did not break in. It did exactly what agents do: it acted on the access it had. The freeze was a sentence in a chat window, not a permission. Nothing technical stood between the agent and the live data. If you are going to build agents, you need to understand the loop that makes them powerful, and you need the guardrails that keep the loop from deleting the database: tools with boundaries, a step that checks the work, a checkpoint that survives a crash, and a human approval before anything irreversible.

This part builds that agent with LangGraph, then measures it instead of admiring it. I ran the same 8 tasks through three versions: a full agent with a verify step, the same agent without it, and a single LLM call with no tools at all. The headline results are blunt. The agent without the verify step answered all 8 tasks correctly in substance. The single LLM call answered none of them correctly, and on one question it confidently invented a formula from gas chromatography. The verify step, the part I expected to help most, made the agent worse on the hardest question. And the whole comparison, 24 agent runs plus 8 single-shot runs, cost about two tenths of a cent. Here is what I built, what broke, and what I would do differently.

Part 6 of the AI Engineer Series. Part 1 mapped the roadmap from 390 real job postings, Part 2 covered the Python that matters for AI work, Part 3 opened the black box of tokens and temperature, Part 4 built RAG from zero, and Part 5 built the eval habit this part depends on. Part 7 connects this agent to real tools with MCP, and Part 8 ships the whole thing to production.

What an agent actually is

An agent is a language model inside a loop, with tools and memory. The model decides what to do next, a tool does it, the result comes back as an observation, and the model decides again. That loop is the whole difference between a chatbot and an agent. A chatbot answers from what it already knows. An agent goes and finds out, then checks whether what it found is enough, then acts again.

The pattern has a name and a paper. ReAct, short for Reasoning and Acting, comes from Yao and colleagues at Princeton and Google in 2022. It interleaves reasoning traces with actions: the model writes a thought, takes an action, reads the observation, and updates its plan. On question answering and fact verification benchmarks, ReAct beat plain chain-of-thought reasoning by grounding the model in observations from a real tool instead of letting it reason only from memory. Every agent framework since then is a variation on this loop. The differences between frameworks are not about the loop. They are about who controls it, what happens when it breaks, and what survives a crash.

Three things decide whether your agent is a demo or a system. First, state: what the agent remembers between steps, and whether that memory survives longer than one request. Second, stop conditions: the maximum steps, the budget, and the rule for giving up gracefully instead of looping until the bill arrives. Third, boundaries: which tools the agent may call, with what permissions, and which actions need a human to say yes first. Most tutorials teach the first half of the loop and skip all three. This part does the opposite.

Why LangGraph, and why now

LangGraph is LangChain's low-level orchestration library for stateful agents, and it reached its stable 1.0 release on 17 October 2025, with 1.2.16 shipping on 16 October 2026. It is MIT licensed, and Klarna, Uber, LinkedIn, Replit, and Elastic run it in production. The installed version behind this article is 1.2.16, so the code below matches what I actually ran.

Three ideas make it different from a hand-rolled loop. First, the agent is a graph, not a script. You define a typed state object, nodes that read it and return partial updates, and edges that route between nodes. Cycles are first-class, so a retry loop is structure you wrote, not behaviour you hope the model exhibits. A conditional edge is a function you can unit test. Second, durable execution. LangGraph writes state to a checkpointer after every node transition, in memory, SQLite, or Postgres, scoped by a thread ID. If the server dies mid-run, the graph resumes from the last completed node instead of starting over. Because every historical state is persisted, you can replay a past run to reproduce a bug exactly, which is the closest thing this field has to a debugger. Third, human-in-the-loop as a primitive. The interrupt() function pauses a run at a node and holds it indefinitely until a human resumes it with Command(resume=...). Approval is not a prompt you add. It is a node the graph cannot pass without you.

My opinion, after building with it: most agent frameworks hide the loop from you, and that is exactly backwards. The loop is the part that breaks in production, so it should be the part you can see, test, and put a limit on. LangGraph shows you the loop. That is its whole case, and it is a strong one.

That said, do not reach for it by default. If one LLM call answers the question, use one LLM call. If the steps are fixed and never branch, write a function. If you want a team of role-playing agents out of the box, CrewAI fits better. If you want the simplest hosted path and you live inside OpenAI's stack, the OpenAI Agents SDK fits better. If type safety is your top concern, look at Pydantic AI. Pick by the failure mode you fear most. If you fear a crash losing an hour of work, or a loop running away, or an action that needs approval, LangGraph is the right default. That is the bet this series makes, and the experiment below tests whether the bet pays.

The fastest way to build one

If you have never built an agent, do not start with a custom graph. Start with the prebuilt ReAct agent. It builds the full loop, LLM node, tool node, conditional routing, in one call:

from langchain_core.tools import tool from langgraph.prebuilt import create_react_agent from langchain_openai import ChatOpenAI @tool def search_docs(query: str) -> str: """Search internal documentation. Use for policy, billing, or feature questions.""" return lookup(query) # your search function here  @tool def calculator(expression: str) -> str: """Evaluate arithmetic. Use for any calculation, e.g. calculator("20.3 / 2.1").""" return str(eval(expression, {"__builtins__": {}}, {})) model = ChatOpenAI(model="gpt-4o-mini", temperature=0) agent = create_react_agent( model=model, tools=[search_docs, calculator], prompt="You are a research assistant. Use tools to look up facts before answering. If you cannot find information, say so. Do not guess.", ) result = agent.invoke({"messages": [("user", "How many times faster were 30 tuned threads than sequential sync?")]}) print(result["messages"][-1].content) 

Two details in that snippet do most of the work. The tool docstrings are not decoration. The model reads them to decide which tool to call, so a vague docstring produces wrong tool calls. Write each docstring as if you were instructing a new colleague: what the tool does, when to use it, and one example. And the prompt line "If you cannot find information, say so" is load-bearing. Without it, the model will answer from memory and sound confident. With it, you get refusals you can trust.

Under the hood, create_react_agent compiles a graph that looks like this: user message, agent (LLM), tool calls, tools, back to the agent, until the model answers directly. That is enough for most first agents. You drop down to a hand-built StateGraph when you need a node the prebuilt agent has no room for: a verify step that checks the evidence before answering, an approval pause before a destructive tool, or a branch on a tool result. My measured agent is a StateGraph for exactly that reason. It has four nodes: reason, which picks the next action; act, which runs the tool; verify, which asks whether the evidence answers the question; and answer, which writes the final response from the evidence only. The verify node is the experiment. Everything else is scaffolding.

The experiment: one agent, three versions, eight tasks

Enough theory. Here is the setup, so you can reproduce it or argue with it. The agent is a LangGraph StateGraph with the four nodes above. Its tools are SEARCH, a TF-IDF search over 10 short passages drawn from facts already published in this series, and CALC, safe arithmetic. To simulate real life, the first SEARCH call in every run fails with a simulated 429 rate limit, and the agent has to retry. The step budget is 6. The model is Gemini Flash-Lite. The 8 tasks mix single-fact questions, two-fact questions that need arithmetic, one unanswerable question, and one question with nothing in the corpus, which is a trap for infinite loops.

I ran three versions on all 8 tasks. Version A is the full agent with the verify node. Version B is the same agent with the verify node removed: reason, act, answer, nothing else. Version C is the control: a single LLM call with no tools and no retrieval, just the question. I graded every answer two ways: token F1 against a reference answer, and a manual read of whether the answer was substantively correct, because Part 5 taught me that F1 alone lies about short answers.

The results

Mean token F1: version A 0.626, version B 0.714, version C 0.181. Substantively correct answers, by my manual read: version B 8 out of 8, version A 7 out of 8, version C 0 out of 8. Mean latency per task: A 20.1 seconds, B 16.2 seconds, C 4.2 seconds. Mean steps per task: A 7.0, B 6.5, C 1.0. Total tool calls across the 8 tasks: A 16, B 20, C 0. The whole comparison used 15,388 tokens and cost about $0.0019 at Flash-Lite list pricing. The agent is roughly five times slower than a single call and roughly four times better, and the cost is rounding error either way.

Three individual results carry the story. First, the single-shot control hallucinated on the RRF question. Asked for the Reciprocal Rank Fusion formula, it answered with the Response Factor formula from gas chromatography, complete with variables for peak areas and concentrations. Same acronym, wrong field, total confidence. That is what answering from memory looks like when the model does not know it does not know. Second, the verify node cost version A its only failure. On the question "how many times faster were 30 tuned threads than sequential sync", version A answered "I don't know" and scored 0. Version B did the division and answered 9.67 times faster, correctly. Third, both agent versions correctly refused the two unanswerable questions, 2 out of 2 each, while the single-shot control invented or hedged. The agents stopped instead of looping: every agent run ended inside the 6-step budget, including the trap question with nothing in the corpus.

What the numbers mean

Six lessons, in the order that will save you the most pain.

1. Tools beat memory, and it is not close. The agent versions scored 0.63 and 0.71 against 0.18 for the single call, and 8 out of 8 and 7 out of 8 substantively correct against 0 out of 8. The gap is not the model. It is the same model in all three versions. The gap is grounding: the agent answers from evidence it retrieved, and the single call answers from weights trained months ago. If your agent's answers matter, give it tools before you give it a bigger model.

2. My verify node made the agent worse. This is the result I did not expect, and it is the most useful one. My verify prompt asked whether the question could be answered "completely and correctly from this evidence alone". On the speed-up question, the evidence held both numbers, 20.3 and 2.1 seconds, but not the ratio. The verify node said no, twice, and the run hit its step budget and abstained. Version B, with no verify node, simply did the division. A verify step that only accepts stated facts punishes every question that needs a derived answer, and derived answers are half of what agents are for. The fix is to verify derivability, not presence: "can the answer be computed from this evidence", or to let the answer node use the calculator. A guardrail that turns hard questions into abstentions is not safety. It is a capability cut dressed as caution.

3. Retries are a feature, not an apology. The first search failed with a simulated 429 on all 16 agent runs. All 16 recovered by retrying the same search, and every one of those runs finished its task. That only works because the tool returns an error string instead of raising an exception. LangGraph stops the loop on unhandled exceptions, so a tool that throws kills the run, while a tool that returns "ERROR: rate limited, retry" gives the model something to reason about. Wrap every tool body in try/except and return the error as text. It is the cheapest reliability win in this whole article.

4. "I don't know" is the most valuable output an agent has. Both agent versions refused both unanswerable questions. That is not a party trick. It is the difference between an agent and a confident liar, and the RRF hallucination above shows what the alternative looks like. Two details matter. First, the refusal only works because the answer node is told to answer from the evidence only, and the evidence is visible in the same prompt. Second, my grader made the same exact-phrase mistake as Part 5: the single-shot control's refusal on the MTEB question scored 0 because it said "I am not sure" instead of "I don't know". Grade refusal by meaning, not by magic string, or you will punish the behaviour you want.

5. Token F1 punishes good answers. On the ticket ID question, both agent versions gave the exactly correct answer, "KB-2024-7831", and both scored 0.333, because F1 counts the extra words in "Based on the provided evidence, vector search misses the exact ticket ID..." as errors. A short correct answer and a long correct answer get different scores for the same substance. Part 5 found the same thing. If you grade agents with token overlap, you will optimise for terse answers instead of correct ones. Grade substance, and read the answers.

6. Loops cost time, not money. The agents took 16 to 20 seconds per task against 4.2 for a single call, but the whole comparison cost $0.0019. At this scale the budget to guard is latency and steps, not dollars. Put a step cap on every agent, report the median latency, and remember that one 31-second outlier moved a mean by seconds in Part 5. The same applies here: a single slow tool call dominates the total, and the user feels it before the bill does.

Crash recovery and human approval, measured

Two more LangGraph features, tested rather than asserted. First, durable execution. I ran the speed-up task on a graph with a SQLite checkpointer, streamed two state updates, then stopped the run cold to simulate a crash. I resumed it on the same thread ID. The resumed run finished the task, and across the crash and the resume the search tool ran exactly twice in total: once failing with the simulated 429, once succeeding. No tool call was repeated, and no completed step was redone. The resumed run's answer repeated the verify-gate weakness from the main experiment: it stated the raw numbers, 20.3 and 2.1 seconds, instead of computing the 9.67x ratio. Durable execution saves your progress. It does not fix your logic.

Second, human-in-the-loop. I compiled a second graph with interrupt_before set to the answer node, ran the chunking question, and the graph paused exactly where I told it to, with the next node showing as "answer" and no answer written. I resumed it with Command(resume="approved") and it produced the correct answer: recursive chunking reached recall@3 of 1.00. That pause is the difference between the Replit story and a safe agent. Lemkin's freeze was a sentence. An interrupt is a node the graph cannot pass. For any tool that deletes, sends, charges, or publishes, put the interrupt before the tool, not before the answer. The demo used 1,810 tokens and cost about $0.0002, bringing the whole article's experiments to 17,198 tokens and about $0.002.

One warning from the framework docs, worth repeating: on resume, LangGraph restarts the whole node, so any side effect placed before an interrupt runs again and must be idempotent. A tool that charges a card and then waits for approval will charge twice. Put the side effect after the interrupt, or make it safe to repeat. This is the kind of detail that only matters once, and then it matters a lot.

Build your first agent this week

Here is the smallest version that works, in five steps.

Step 1: pick one task with a verifiable answer. Not "help me with research". Something like "answer refund-policy questions from our docs, and cite the passage". If you cannot write the reference answer, you cannot tell whether the agent works.

Step 2: give it two tools, maximum. One search, one calculator, or one lookup. Every tool you add is a decision the model can get wrong. Two tools that work beat ten tools that confuse it.

Step 3: set a step budget and a latency budget. Six steps and a timeout per tool call is a good start. A loop without a budget is a billing incident waiting for a trigger, and the trigger is usually a question nobody tested.

Step 4: add one eval before adding a third tool. Take the 20-case golden set from Part 5 and run it on every change. The eval is what tells you whether the verify node helped or hurt, and I only know mine hurt because I measured it.

Step 5: put an interrupt before anything irreversible. Delete, send, charge, publish: if the tool cannot be undone, a human approves it. And the agent that runs code or calls tools belongs in a sandbox, not on your laptop with full permissions.

My opinion: ship the two-tool agent with evals before the ten-tool agent without them. The two-tool agent is boring, measurable, and safe. The ten-tool agent is a demo that deletes a database on day nine.

What agents still cannot do

Three honest limits. First, agents fail on the tasks your evals never covered. OpenAI's sycophancy incident from Part 5 is the proof: a large eval suite, green metrics, and a shipped regression, because nobody had written an eval for the exact behaviour that changed. Your agent has the same blind spot. The defence is the same habit: every time a user reports a failure your evals missed, add that failure to the golden set before you fix the bug.

Second, tool outputs are untrusted input. An agent that reads a web page, a file, or a database row is reading text an attacker may have written. That text can carry instructions, and a model that follows instructions will follow those too. This is prompt injection, and it is the reason Part 7 exists: connecting agents to real tools with MCP is where the attack surface opens up. Until then, treat every tool result as data, never as orders, and never give an agent a tool whose output you have not seen.

Third, the loop is unbounded by default, and the cost of an unbounded loop is not just money. Gartner predicts that by 2027, four out of ten companies will demote or retire autonomous agents because of governance flaws discovered only after a production incident. A 2025 SailPoint survey of 353 IT professionals found 80 percent of organizations said their agents had already performed unintended actions. The Replit database and the PocketOS volume, deleted by a Cursor agent through a Railway API call in about nine seconds in April 2026, are the public examples. The pattern in every one of them is the same: the agent had the permission, and the only guardrail was a sentence. Permissions beat prompts. Build the permission first.

What is Next in This Series

• Part 1: AI Engineer Roadmap 2026: What 390 Job Postings Actually Require

• Part 2: Python for AI Engineers: Asyncio, APIs, and What Tutorials Skip

• Part 3: How LLMs Actually Work: Tokens, Context, and Temperature

• Part 4: RAG from Zero: Embeddings, Vector DBs, and Retrieval

• Part 5: Evals: The Skill 56% of Jobs Want

• Part 6: Building Your First AI Agent with LangGraph (this article)

• Part 7: MCP: Connecting Agents to Real Tools Safely

• Part 8: LLMOps: Shipping AI to Production

Part 7 is where this agent meets the real world. MCP connects agents to real tools: file systems, databases, APIs, and other people's servers. That is also where prompt injection lives, which is why the series covers tool poisoning and hardening before it covers building. Until then, keep your agent's tools few, its steps capped, and its destructive actions behind a human approval. The loop is powerful. The guardrails are what make it safe to use.

Originally published on LogicStack. Run the code examples right in your browser with our free interactive Python and SQL playgrounds — no setup, no sign-up.