Agent Memory & Context Graphs · 2 of 5
The Context Graph: Wiring Repos, Policies, and Identity
Memory is what you store; context is what you show. A context graph is the relationship layer that makes the choice smart enough for security.
Two ideas are easy to conflate. Memory is everything an agent has stored. Context is the specific slice of it you place in front of the model for one step. The bridge between them is a context graph: a map of how things relate, so that when the agent needs context, it can pull not just a matching chunk but the connected facts that make that chunk meaningful. Without the graph, retrieval returns isolated fragments. With it, retrieval returns a small, connected picture the model can actually reason over safely.
Memory is stored, context is shown
Memory holds everything: episodes, facts, procedures, across sessions. Context is the deliberate subset shown to the model for a particular step. Confusing the two leads either to dumping all of memory into the prompt, which drowns the model, or to showing a lone fragment that lacks the surrounding facts to be understood correctly.
The interesting work is in choosing the subset well. That choice is only as good as the structure you have for reasoning about what relates to what.
The graph in the middle
A context graph connects the entities an agent reasons about and labels the relationships between them: a service contains endpoints, an endpoint is guarded by an auth policy and touches a data class, a service is owned by a team, a tool runs as an identity that holds permissions.
Those edges are what turn a pile of facts into knowledge. Retrieving one node lets you pull its neighbors, so the model sees a fact together with the things that qualify it.
What a security graph connects
For security reasoning, the graph is worth its weight when it wires the right things together. The point is that a finding is not just a finding; it is a finding on an endpoint, in a service, touching a data class, governed by a policy, owned by a team.
These connections are what let an agent judge severity and next steps rather than staring at an isolated alert.
- Repository to service to endpoint to the auth that guards it.
- Tool to the identity it runs as, and the permissions that identity holds.
- Scanner rule to the code sink it flags, and the policy that governs it.
- Finding to acceptance criteria, test evidence, owner, and environment.
Assembling context by traversal
With a graph, assembling context becomes a traversal. Start from what the step is about, walk the edges to the related nodes that matter, gather that connected neighborhood, and hand the model a small, coherent picture instead of a random chunk.
This is what makes retrieval smart: not just similarity to a query, but the structure that says which nearby facts are relevant to this decision.
Why a single chunk is not enough
A lone retrieved passage is rarely enough for a sound security judgment. Knowing that an endpoint exists is different from knowing it changes account data, sits behind a specific auth policy, and is owned by a team with an on-call rotation.
The graph supplies exactly that qualifying context. It is the difference between the model guessing from a fragment and reasoning from a connected, accurate picture.
Keeping the graph current
A context graph is only trustworthy if its edges reflect reality. Code moves, owners change, policies update, and a stale edge can point the model at an assumption that no longer holds. So the graph needs the same freshness discipline as any other memory: edges stamped, refreshed, and expired when the underlying truth changes.
An out-of-date relationship is not a harmless imperfection; it is a confident, connected wrong answer waiting to happen.
What a practitioner decides
Building a context graph means deciding which entities and relationships matter for your agent's decisions, how retrieval traverses them to assemble context, and how the edges are kept current. It is more work than plain similarity search, and it is what lets an agent reason about a system rather than a snippet.
The question is not merely did we retrieve something relevant. It is: did we retrieve it with the connected facts that make it safe to act on?
Key takeaways
- Memory is everything stored; context is the connected subset you show for one step.
- A context graph labels the relationships between the entities an agent reasons about.
- Edges turn isolated facts into knowledge by letting retrieval pull qualifying neighbors.
- For security, wire findings to endpoints, services, data classes, policies, and owners.
- Assemble context by traversal, not just similarity, for a coherent picture.
- Keep edges fresh; a stale relationship is a confident, connected wrong answer.
Glossary
- Context graph
- A map of entities and their labeled relationships used to assemble connected, relevant context.
- Entity
- A thing an agent reasons about, such as a service, endpoint, tool, identity, or finding.
- Edge
- A labeled relationship between two entities, like guarded-by or runs-as.
- Traversal
- Walking a graph's edges from a starting node to gather connected, relevant neighbors.
- Retrieval
- Fetching relevant material to include as context, ideally guided by graph structure, not just similarity.
- Data class
- A label describing how sensitive a piece of data is and how it must be handled.
- Neighborhood
- The set of nodes connected to a given node, which supplies its qualifying context.