Graph agents and when the graph earns its keep

Two constraints made this decision for me, and neither was mine to argue with: runs were long and expensive enough that a failure halfway through could not mean starting over from the first tool call, and a human had to be able to approve an action mid-run before it went any further. The delivery date was fixed as well, which ruled out finding out the hard way and rebuilding later. So I built the same agent twice against those constraints, once as a plain Python while loop and once on LangGraph. The loop was about ninety lines shorter, only one of the two survived being killed mid-run, and understanding why took most of April, because it came down to one architectural fact I had skimmed straight past in the docs.

Why the DAG mental model fails

Almost every diagram of a LangGraph agent looks like a flowchart, so you assume the runtime is a DAG executor: topologically sort the nodes, run them in order, done. That model is wrong, and every confusing thing about the framework follows from it being wrong.

LangGraph's execution engine is Pregel, the same message-passing model Google published for large-scale graph processing. Execution proceeds in super-steps. A node activates when it receives a message, runs, writes its state updates, and votes to halt. The run terminates when all nodes are inactive and no messages are in transit. Nodes that fan out in a single super-step run in parallel with each other. Nodes that run in sequence span separate super-steps.

Hold onto that and a pile of confusing behaviour stops being confusing. Parallel writes to the same state key conflict because they happen inside one super-step, not because of some race in your code. A whole node re-runs on resume, rather than resuming mid-function, because the super-step is the atomic unit of progress. And recursion_limit counts super-steps, so it has almost nothing to do with how many LLM calls you made. A single super-step can contain four parallel nodes, each making three model calls.

Once I stopped reading the graph as a flowchart and started reading it as a scheduler, I could predict its behaviour instead of discovering it.

The state schema is the real API

You define state as a TypedDict, and the annotations on that dict do far more work than the node functions do.

# graph.py
import operator
from typing import Annotated, TypedDict
 
from langchain_core.messages import AnyMessage
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
 
 
class ResearchState(TypedDict):
    """State carried across every super-step of the research graph.
 
    Every key in here is checkpointed on each super-step, so anything
    that does not need to survive a resume should not live here.
    """
 
    messages: Annotated[list[AnyMessage], add_messages]
    findings: Annotated[list[str], operator.add]
    question: str
    verdict: str
 
 
class ResearchInput(TypedDict):
    """What a caller is allowed to pass in."""
 
    question: str
 
 
class ResearchOutput(TypedDict):
    """What a caller gets back, which is far less than the full state."""
 
    verdict: str
    findings: list[str]
 
 
def plan(state: ResearchState) -> dict:
    """Turn the question into a search plan, recorded as a message."""
    ...
 
 
def search(state: ResearchState) -> dict:
    """Run one retrieval pass and append whatever it turned up."""
    ...
 
 
def critique(state: ResearchState) -> dict:
    """Decide whether the findings so far actually answer the question."""
    ...
 
 
def route(state: ResearchState) -> str:
    """Edge function. Loop back into search, or stop."""
    return END if state["verdict"] == "sufficient" else "search"
 
 
builder = StateGraph(
    ResearchState,
    input_schema=ResearchInput,
    output_schema=ResearchOutput,
)
builder.add_node("plan", plan)
builder.add_node("search", search)
builder.add_node("critique", critique)
builder.add_edge(START, "plan")
builder.add_edge("plan", "search")
builder.add_edge("search", "critique")
builder.add_conditional_edges("critique", route, ["search", END])
graph = builder.compile()

Two things in there are load-bearing. The input_schema and output_schema arguments let the graph carry a fat internal state while exposing a narrow contract, which matters more than it sounds like it does once other code starts calling your graph. And .compile() is mandatory. The builder is not runnable; compilation is where the channel wiring and validation happen.

The annotations are reducers. add_messages merges message lists by id, so re-emitting a message updates it rather than duplicating it. operator.add concatenates. A key with no reducer gets last-write-wins semantics, which is fine right until two nodes write it in the same super-step.

The reducer is a concurrency policy, not a fix

Here is the failure that taught me the most. Fan two nodes out of START and have both write the same unreduced key.

# fanout_conflict.py
from typing import TypedDict
 
from langgraph.graph import START, StateGraph
 
 
class State(TypedDict):
    """Deliberately reducer-free, which is the entire point of the example."""
 
    topic: str
    notes: list[str]
 
 
def west(state: State) -> dict:
    """One of two nodes activated in the same super-step."""
    return {"notes": ["west says p99 looks fine"]}
 
 
def east(state: State) -> dict:
    """The other one. Same key, same super-step, different opinion."""
    return {"notes": ["east says p99 does not look fine"]}
 
 
builder = StateGraph(State)
builder.add_node("west", west)
builder.add_node("east", east)
builder.add_edge(START, "west")
builder.add_edge(START, "east")
graph = builder.compile()
 
graph.invoke({"topic": "p99", "notes": []})
# langgraph.errors.InvalidUpdateError: At key 'notes': Can receive only one
# value per step. Use an Annotated key to handle multiple values.

The docs file that under the error code INVALID_CONCURRENT_GRAPH_UPDATE, and the fix everyone reaches for is to slap Annotated[list[str], operator.add] on the key. It makes the error go away.

It does not make the problem go away. Adding a reducer chooses a policy. Before the reducer, two concurrent writers to one key were a hard error, loudly, at the exact moment of conflict. After the reducer, they are an order-dependent result. operator.add concatenates in whatever order the writes land within the super-step, so the contents of notes now depend on which node finished first, and nothing in the type signature says so. You converted a crash into a silent ordering dependency and called it a fix.

That is worth internalising because it generalises. A reducer is concurrency control. Picking operator.add says "order does not matter to me". Picking a custom reducer that sorts by a stable key, or that merges by node id, says something stronger and usually more accurate. If you cannot say out loud what your reducer's conflict policy is, you have not resolved the race, you have hidden it.

I now write custom reducers for anything two nodes can touch, even trivial ones, purely so the policy is written down somewhere.

Send, for the fan-out you actually meant

When the fan-out width is dynamic, static edges cannot express it. Send can. Each Send carries its own private payload to the target node, so the workers do not all read the same shared blob.

# fanout_send.py
import operator
from typing import Annotated, TypedDict
 
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
 
 
class MapState(TypedDict):
    """Parent state. `summaries` is the only key workers write."""
 
    documents: list[str]
    summaries: Annotated[list[str], operator.add]
    report: str
 
 
class WorkerState(TypedDict):
    """Private per-Send payload. Never merged back except via the reducer."""
 
    document: str
 
 
def dispatch(state: MapState) -> list[Send]:
    """Fan out one worker per document inside a single super-step."""
    return [Send("summarise", {"document": doc}) for doc in state["documents"]]
 
 
def summarise(state: WorkerState) -> dict:
    """Summarise exactly one document. Runs in parallel with its siblings."""
    return {"summaries": [f"summary of {state['document'][:40]}"]}
 
 
def reduce_report(state: MapState) -> dict:
    """Collapse every worker summary into one report."""
    return {"report": "\n".join(state["summaries"])}
 
 
builder = StateGraph(MapState)
builder.add_node("summarise", summarise)
builder.add_node("reduce_report", reduce_report)
builder.add_conditional_edges(START, dispatch, ["summarise"])
builder.add_edge("summarise", "reduce_report")
builder.add_edge("reduce_report", END)
graph = builder.compile()

All those summarise invocations land in one super-step. Their writes to summaries go through operator.add, which is exactly the situation from the previous section, and here I genuinely do not care about ordering because the reduce step joins them. That is the difference between choosing a policy and inheriting one.

Command, and keeping the graph drawable

Command lets a node update state and pick its own next hop in one return value. The interesting part is the type hint.

# supervisor.py
from typing import Literal, TypedDict
 
from langgraph.graph import END, START, StateGraph
from langgraph.types import Command
 
 
class TeamState(TypedDict):
    """Shared state for a supervisor and two workers."""
 
    task: str
    draft: str
    review_notes: str
 
 
def supervisor(state: TeamState) -> Command[Literal["writer", "reviewer", "__end__"]]:
    """Route the task and record the routing decision in one step.
 
    The Literal in the return annotation is what lets the graph be
    rendered statically. Drop it and the visualiser sees a dead end.
    """
    if not state["draft"]:
        return Command(update={"task": state["task"].strip()}, goto="writer")
    if not state["review_notes"]:
        return Command(goto="reviewer")
    return Command(goto=END)
 
 
def writer(state: TeamState) -> Command[Literal["supervisor"]]:
    """Produce a draft, then hand control back up."""
    return Command(update={"draft": "..."}, goto="supervisor")
 
 
def reviewer(state: TeamState) -> Command[Literal["supervisor"]]:
    """Review the draft, then hand control back up."""
    return Command(update={"review_notes": "..."}, goto="supervisor")
 
 
builder = StateGraph(TeamState)
builder.add_node("supervisor", supervisor)
builder.add_node("writer", writer)
builder.add_node("reviewer", reviewer)
builder.add_edge(START, "supervisor")
graph = builder.compile()

Command also takes graph=Command.PARENT to hand control back out of a subgraph, and resume= to feed a value into an interrupted run. That resume= path is the human-in-the-loop story, and it's the one capability that genuinely has no clean equivalent in a while loop.

One trap here that the docs warn about and I still walked into: if a node has a static add_edge leaving it and also returns Command(goto=...), both fire. The downstream node executes twice. That cost me a duplicated paid tool call inside a run that someone was waiting on the output of, plus the afternoon it took to work out that the framework had done exactly what it documents.

Durability, limits, and what they cost

Persistence has three modes, typed as Durability = Literal['sync', 'async', 'exit']. Sync writes the checkpoint before the next step starts. Async writes it while the next step is already executing. Exit persists only when the run exits. The docs I read do not state which one is the default, so I won't either; set it explicitly and you never have to care.

# run.py
from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.errors import GraphRecursionError
 
from graph import builder
 
 
def run(question: str, thread_id: str) -> dict:
    """Run one research thread with sync durability and a tight step budget.
 
    recursion_limit counts super-steps, not model calls. The default of
    1000 is a runaway-protection number, not a budget.
    """
    with SqliteSaver.from_conn_string("checkpoints.db") as saver:
        graph = builder.compile(checkpointer=saver)
        config = {
            "configurable": {"thread_id": thread_id},
            "recursion_limit": 24,
        }
        try:
            return graph.invoke({"question": question}, config, durability="sync")
        except GraphRecursionError:
            snapshot = graph.get_state(config)
            return {
                "verdict": "budget_exhausted",
                "findings": snapshot.values.get("findings", []),
            }

The default recursion_limit is 1000 super-steps, and blowing through it raises GraphRecursionError. Treat that as a smoke alarm rather than a budget. A two-node ping-pong between a planner and a critic will spend real money long before it reaches 1000, and it will do so cheerfully, because nothing in the graph knows the difference between progress and oscillation. The graceful alternative is the managed RemainingSteps channel, which lets a node see how much budget is left and wrap up deliberately instead of dying at the limit.

What the state actually costs

The costs I did not anticipate are mostly about state. Everything in state is checkpointed every super-step, so checkpoint size scales with how much you're carrying rather than with how many decisions you've made. add_messages grows monotonically by design, so a long agent thread means every super-step writes an ever-larger blob. Two hundred super-steps over a fat state is a lot of writes for very little information, and the write amplification is invisible until you look at the size of the checkpoint table.

What fixed it for me was being ruthless about what deserves to be in state at all. Retrieved document text moved to a content store, with state holding only the ids. Intermediate scratch values that no downstream node reads got deleted rather than left lying around because they were convenient during development. The rule I ended up with is that a key belongs in state only if a node that runs after a resume needs to read it. Everything else is a local variable that happens to be in the wrong scope.

And one sharp edge with no workaround I know of: renaming a state key silently loses the persisted value on existing threads. The old key is still in the checkpoint, the new key reads as absent, nothing errors, and the graph carries on with a default. I found that out on threads that were already in flight, which is the worst way to find it out. Treat the state schema like a database migration, because that is what it is.

Reading a run after the fact

Debugging is different in a way nobody warns you about. There is no stack trace that spans the run, because the run is not a call stack. It's a sequence of super-steps, and by the time you notice something is wrong the frames you'd want to inspect were discarded several steps ago. get_state_history() is the actual debugger, and learning to read it is a skill you have to deliberately acquire rather than something you pick up.

The workflow that replaced breakpoints for me is dull and effective. Reproduce with a fixed thread_id, walk the state history backwards until the state stops looking right, then note the super-step where a key first held a value nobody expected. That's your failing node, and it is frequently not the node that raised. Once I stopped hunting for a stack trace and started reading history as a diff over state, the framework got a lot less mysterious.

Alternatives that occupy different ground

Three alternatives are worth knowing precisely, because they occupy different points rather than competing on the same one.

The OpenAI Agents SDK has no graph at all. Delegation is a tool call: handing off to another agent surfaces as a tool named transfer_to_<agent_name>, and the model picks it the same way it picks any other tool. Control flow is whatever the model decides, bounded by max_turns, which raises MaxTurnsExceeded. Much less machinery, much less say over the path.

Pydantic AI's pydantic-graph is a typed state machine. Nodes are dataclasses subclassing BaseNode[StateT, DepsT, RunEndT], and the edge set is the return type annotation of run(). You return the next node instance, and termination is End(value). The type checker validates your control flow, which is a real advantage over a graph assembled by string node names. Their own docs say graphs are "not the right tool for every job", which I appreciated.

Temporal-style durable execution sits on a different axis entirely. Workflow code is deterministic and gets replayed from an event history; activities execute once and have their results recorded. The line worth drawing is this: a checkpointer stores state snapshots, while deterministic replay stores the decision log. Snapshots are easy to reason about and expensive when state is large. Replay is cheap to store and imposes real determinism constraints on your code. Neither is strictly better, and knowing which one you need is more useful than knowing either framework's API.

What I'd check before reaching for a graph

The graph earned its keep in exactly one of my two builds, and only because that build needed cycles, durability, and human approval mid-run at the same time. Take away any one of those and the plain loop wins.

Cycles alone are a while loop. Durability alone is a job queue with a status table. Human-in-the-loop alone is a state machine with a pause. It's the combination that gets ugly to hand-roll, because pausing a cyclic process at an arbitrary point and resuming it later on a different machine is the thing the checkpointer and the super-step boundary actually solve.

Everything else I built was a workflow with fixed control flow and a model call at each step, and Anthropic's own guidance says the same thing in blunter terms: start with the simplest thing, and add agentic machinery only when the simpler approach falls short. Most production systems are workflows wearing agent branding. Mine were.

If you're on the fence right now, do this before you pick: write down the longest path your system takes, and count how many of the steps on it are decisions the model makes versus steps you already know the order of. If the answer is mostly the latter, write the loop. If you find yourself needing to interrupt that path, get a human to approve something, and come back to it an hour later on a different process, compile the graph and set durability="sync" on day one, because retrofitting persistence onto a state schema with live threads is the migration you don't want.