Most agent frameworks I have used start from the same mental model: a chain. A planner hands off to a researcher, the researcher hands off to a writer, the writer hands off to a reviewer, and the output falls out the end. It is an appealing picture because it maps onto how we describe work to each other. It is also the wrong abstraction for almost every task I have actually needed agents to do. This post is about why, and about the model we ended up with in GraphBit instead.
Where the chain breaks
A chain encodes three assumptions at once. It assumes the order of steps is known before execution starts. It assumes each step has exactly one successor. And it assumes control flows forward only, so that nothing upstream ever needs to be revisited.
Real tasks violate all three within minutes. A research agent discovers that the question was ambiguous and needs the planner to disambiguate it. A tool call fails and the right response is to try a different tool, not to pass the failure downstream. Two sub-questions are independent and could be answered in parallel, but the chain forces them to run in sequence. A reviewer rejects a draft, and the honest fix is to loop back to the writer with feedback rather than to bolt a second reviewer onto the end.
Framework authors know this, which is why chains grow escape hatches: conditional edges, retry wrappers, “supervisor” agents that route dynamically, and callbacks that mutate the pipeline while it runs. Each patch works locally. Collectively they turn the chain into an implicit graph whose structure lives in scattered control-flow code, where it cannot be inspected, visualised, tested, or reasoned about as a whole.
Make the graph explicit
The alternative is to admit that the structure is a graph and to make that graph the first-class object the framework executes. In GraphBit, agents, tools, and memory stores are all nodes. Edges carry data and control. A workflow is a directed graph, and running a workflow means traversing that graph under a scheduler that understands dependencies.
Once the graph is explicit, the escape hatches become ordinary features:
- Branching is a node with more than one outgoing edge and a predicate deciding which edges fire.
- Parallelism is two nodes with no dependency between them. The scheduler runs them concurrently without anyone writing async code.
- Cycles are edges that point backwards. A reviewer that rejects a draft simply routes to the writer again, with a guard on the cycle count so the loop terminates.
- Fan-in is a node with several incoming edges that waits for all, or any, of them before executing.
None of this is exotic from a systems point of view. Dataflow engines and build systems have executed dependency graphs for decades. What is different in the agent setting is that the nodes are non-deterministic, expensive, and slow, and the graph topology itself may depend on what a node produced. That combination drives the rest of the design.
Non-determinism changes what “done” means
A build system can assume that running the same node twice with the same inputs gives the same output. An LLM agent cannot. Two consequences follow.
First, the scheduler has to record what each node actually emitted, not just that it finished. Downstream routing decisions depend on content, so the execution trace must be a first-class artefact. This is also what makes debugging possible: when a workflow goes wrong, you replay the graph with the recorded outputs and inspect the exact edge where it diverged from what you expected.
Second, validation belongs at the edges. A chain tends to validate at the end, when the final answer is checked against a schema or a rubric. In a graph, each edge can carry a contract: the type of data it expects, a schema the payload must satisfy, or a predicate over the payload. A node that produces malformed output fails at the edge, before its output contaminates the rest of the run. In practice this catches a large fraction of agent failures early and locally.
Dynamic topology without losing structure
The hardest design decision was how much to let the graph change while it runs. A fully static graph is easy to reason about but cannot express “spawn one worker per sub-question the planner produced.” A fully dynamic graph, where any node can add arbitrary nodes and edges, is just the chain-with-escape-hatches problem again.
The compromise that worked is bounded expansion. Nodes can be declared as templates that expand into a fixed pattern at runtime, such as a map over a list the upstream node produced, or a retry sub-graph around a fragile tool. The pattern is fixed at design time. Only its multiplicity is decided at runtime. This keeps the graph inspectable before execution while allowing the shape of the work to follow the shape of the data.
Memory as a node, not a side effect
In chain-based frameworks memory is usually ambient: a shared dictionary every step reads and writes, or a vector store agents query through a tool. Both approaches hide dependencies. If a downstream agent’s behaviour depends on what an upstream agent wrote into memory, that dependency should be visible in the graph.
Treating memory stores as nodes does exactly that. A write to memory is an edge into the memory node. A read is an edge out of it. The scheduler can now order operations correctly, run reads in parallel safely, and show in the trace precisely which memory state a given agent saw. It also makes it natural to have several memory nodes with different characteristics, such as a fast transient store for the current run and a persistent store that outlives it, without agents needing to know which is which.
What this buys in practice
The concrete payoff shows up in three places.
Reliability. Edge contracts and bounded cycles turn most failure modes into local, recoverable events rather than end-of-run surprises.
Latency. Independent nodes run concurrently by default. In workflows with any real fan-out, this alone recovers a large share of the wall-clock time a sequential chain wastes.
Legibility. The workflow is a picture. You can draw it, diff it between versions, and point at the node that misbehaved. When a team is iterating on an agent system, this matters more than any individual feature.
Open questions
Two problems remain unsolved to my satisfaction. The first is cost control under cycles: a loop guarded by a maximum iteration count still burns tokens on every pass, and better stopping criteria that look at whether the loop is making progress are needed. The second is composition. Sub-graphs should be reusable across workflows the way functions are reusable across programs, with clean interfaces, and the type system for edges is not yet rich enough to make that ergonomic.
The broader claim of this post does not depend on either. If the work you want agents to do has branches, independent parts, or feedback, then the structure of that work is a graph. A framework that pretends otherwise will make you write the graph yourself, badly, in the gaps between its abstractions.