Grounding Agents in a Large Codebase — AI Coding Assistants II

A software agent is only useful in a large codebase if it can find the right facts before it acts. Without grounding, it guesses from fragments: a file name, a familiar pattern, a misleading comment, or a stale memory of how similar systems work. Grounding is the discipline of connecting the agent to the actual repository, the current task, the build system, the tests, and the rules of the team, so its answers and code changes are based on evidence rather than confidence alone.

Why large codebases confuse agents

A large codebase is not just a pile of files. It is a living system of modules, dependencies, conventions, generated artifacts, build steps, feature flags, tests, deployment rules, and team habits. Humans learn this system slowly by reading code, asking colleagues, breaking things, and remembering what happened. An agent has none of that history unless we provide it.

A large language model, often shortened to LLM, predicts useful text from patterns it has learned. That makes it good at explaining common designs and writing plausible code. It does not automatically know the private code in your repository, the branch you are on, the recent refactor that changed a public interface, or the fragile test that guards a production incident. If those facts are not placed within reach, the model may produce an answer that sounds right and fails in practice.

Grounding starts with a simple idea: before an agent writes, it must look. The quality of that looking determines the quality of the work.

⚠️

Confidence is not evidence. An agent can sound certain while using incomplete or outdated context, so every important claim should trace back to code, tests, documentation, or tool output.

What grounding means

Grounding is the process of attaching an agent's reasoning to reliable sources. In a codebase, those sources include source files, dependency manifests, type definitions, configuration, documentation, issue history, test results, and command output from the local development environment or continuous integration system.

Grounding is not the same as giving the agent the whole repository. Most models have a context window, which is the amount of text they can consider at one time. Even when that window is large, filling it with everything creates noise. Good grounding is selective. It brings in the smallest set of facts that can answer the question or support the change.

A grounded agent should be able to explain why it touched a file, which evidence it used, what it did not inspect, and how it checked the result. That explanation is not decoration. It is how a human reviewer can distinguish informed work from lucky output.

Build a map before asking for changes

The first useful artifact is a map of the codebase. A map does not need to understand every line. It should answer basic navigation questions: where are the entry points, which modules depend on which other modules, where are tests located, how are packages built, and which files are generated or should not be edited by hand.

This map can combine several views. A file tree shows location. A dependency graph shows relationships. An abstract syntax tree, which is a structured representation of code, can show functions, classes, imports, and call sites more accurately than plain text search. Build metadata shows what commands turn code into a working system.

The map should be refreshed when the repository changes. A stale index can be worse than no index because it gives old facts a new appearance. In fast-moving repositories, grounding should prefer the checked-out branch and recent tool output over cached summaries.

Identify entry points such as applications, services, command-line tools, and scheduled jobs.
Track dependency manifests, package boundaries, and build targets.
Separate generated code, vendored code, test fixtures, and hand-written source.
Record ownership hints such as module maintainers, codeowners files, and review paths.
Keep summaries tied to commit identifiers so the agent can detect stale knowledge.

A useful map combines a file tree, a dependency graph, a syntax tree, and build metadata.

Retrieve evidence, not just matching words

When a user asks an agent to change behavior, the agent must retrieve relevant evidence from the repository. The simplest method is keyword search. It works well for exact names: a function, an error message, a configuration key, or a test name. It works poorly when the user describes intent in different words from the code.

Embedding search helps with that gap. An embedding is a numeric representation of text that makes similar meanings easier to compare. It can find code or documentation that is conceptually related to a request even when the same words are not used. But embeddings are not magic. They can retrieve nearby-looking material that is not actually relevant, especially in repositories with repeated patterns.

The best retrieval systems combine methods. They search exact terms, inspect symbols, follow imports, read tests, and rank results by proximity to the task. They also keep snippets small enough to fit in the context window while preserving the surrounding definitions needed to understand them.

Keyword search finds exact names while embedding search bridges different wording; the best systems combine both.

Let tools close the loop

Reading code is only half of grounding. The agent also needs feedback from tools that execute against the real repository. A type checker can reveal an invalid call. A linter can catch style and safety rules. A unit test can show that behavior changed. A build can expose missing generated files or broken packaging. A local command can surface facts no static index would know.

This turns the agent from a one-shot writer into a worker in a loop: inspect, plan, edit, run checks, interpret failures, and adjust. The loop should be explicit. If tests fail, the agent should report the failure and decide whether it is related to the change. If a command is unavailable or too expensive, it should say so rather than pretending validation happened.

Tool use also creates better context. A compiler error is dense, current, and directly tied to the branch. A failing test gives a concrete behavior to reason about. These signals are often more valuable than another page of source code.

✅

Validation is context. A fresh test result is one of the strongest grounding signals because it comes from the codebase as it exists now.

Tools turn the agent into a worker in a loop: inspect, edit, run checks, and interpret the results.

Protect the codebase from the agent

Grounding improves capability, but capability needs boundaries. An agent that can read secrets, change deployment files, run destructive commands, or send code to external services may create risks that have nothing to do with code quality. Safe grounding includes permission design.

Access should follow least privilege, which means the agent gets only the permissions needed for the task. Read access and write access should be separate. Running tests should not imply permission to deploy. Editing application code should not imply permission to change identity, networking, billing, or production configuration.

Identity and audit matter. If an organization uses Microsoft Entra ID or another identity provider, agent actions should still be attributable to an accountable user, service identity, or workflow. Logs should show what the agent read, what it changed, what commands it ran, and which human approved the result.

Block access to secrets, private keys, tokens, and customer data unless a task explicitly requires a controlled path.
Require approval before modifying deployment, identity, data migration, or infrastructure files.
Run commands in a sandbox when possible, with network and file-system limits appropriate to the task.
Record prompts, retrieved files, tool calls, diffs, and test results for review and incident analysis.
Use repository policies so agent-generated changes pass the same checks as human changes.

⚠️

Do not ground on secrets. Secrets may help an agent complete a task, but exposing them to a model or log often creates a larger security problem than the task itself.

Grant an agent only the access a task needs, with scoped audited permissions, and deny the rest.

The decisions leaders and practitioners face

Grounding is not only an engineering detail. It shapes cost, speed, risk, and trust. Leaders must decide which repositories are ready for agents, which tasks are allowed, which checks are mandatory, and who owns the failures. Practitioners must decide how much context to retrieve, when to ask a human, and when to stop because the evidence is weak.

A common mistake is to start with the most ambitious goal: let the agent handle broad feature work across many services. A better starting point is narrow, observable work: update tests after an interface change, explain a module, draft a migration plan, fix a well-scoped lint issue, or propose a small refactor with a complete diff. These tasks are easier to ground and easier to review.

Platform choices matter too. Whether a team uses Microsoft Foundry, an internal agent framework, or another platform, the important questions are the same: what sources can the agent read, how are indexes refreshed, which tools can it run, how are permissions enforced, and how can humans inspect the evidence behind its work?

Measure grounding as an engineering system

Grounding should be measured, not admired. Useful measures include whether the agent retrieved the files a senior engineer would expect, whether it cited evidence accurately, whether its changes passed tests, how often humans rejected its reasoning, and how much time review actually saved. A fast agent that creates slow reviews is not improving the system.

Teams should keep a small set of benchmark tasks from real work. Each task should have expected evidence, acceptable changes, and required checks. Run the agent against these tasks when indexes, prompts, models, permissions, or tools change. This turns agent quality into something closer to a regression test than a mood.

The goal is not to remove uncertainty. Software work always contains uncertainty. The goal is to make uncertainty visible early, before the agent edits the wrong layer, trusts the wrong file, or hides a weak assumption inside a confident answer.

📌

Track rejection reasons. Human review comments are valuable training data for the grounding system because they reveal missing evidence and poor task boundaries.

Key takeaways

Grounding connects an agent to current code, tests, tools, documentation, and policies so it works from evidence.
Do not give an agent the whole repository by default; retrieve the smallest useful context and preserve the links between evidence and claims.
A codebase map, symbol-aware search, dependency information, and fresh tool output are stronger together than any single method.
Tool feedback such as builds, type checks, linters, and tests is part of grounding, not a final formality.
Permissions, audit logs, and human approvals are necessary because better-grounded agents can also do more damage.
Start with narrow, reviewable tasks and measure whether grounding improves real engineering outcomes.

Glossary

Agent: A software system that uses a model, tools, and instructions to plan and carry out tasks on a user's behalf.
Grounding: The practice of tying an agent's reasoning and actions to reliable, task-relevant sources of evidence.
Large language model: A model trained on large amounts of text and code to predict and generate useful language-like output.
Repository: A version-controlled collection of source code, tests, configuration, documentation, and history.
Context window: The amount of text and other input a model can consider at one time.
Embedding: A numeric representation of text or code that helps software compare items by meaning rather than only by exact words.
Dependency graph: A map showing which modules, packages, or services rely on which others.
Abstract syntax tree: A structured representation of code that shows its grammatical parts, such as functions, classes, imports, and expressions.
Least privilege: A security principle that gives a user, service, or agent only the access needed to perform a specific task.