Context Engineering — Building AI Apps: Retrieval

A large language model can sound informed even when it is missing the facts it needs. That is not magic, and it is not malice. The model is doing what it was built to do: using the text and other signals available to it to predict a useful next response. Context engineering is the discipline of shaping those available signals so the system has a fair chance of being correct, safe, and relevant.

Start with the machine in front of you

A large language model, or LLM, is software trained to generate text by finding patterns in examples. When you ask it a question, it does not open a private library in its mind and look up the truth. It receives an input, breaks that input into pieces called tokens, and produces an output based on what its training and current input suggest should come next.

The current input matters enormously. It can include your instruction, the conversation so far, retrieved documents, database results, tool outputs, user profile details, policy rules, and formatting requirements. All of that is context. Context engineering is the deliberate design of this input environment.

A simple prompt asks for an answer. Context engineering asks a broader question: what must be true around this model call for the answer to be useful? That includes what information to include, what to leave out, how to order it, how to label it, how to verify it, and when to refresh it.

Context is not just a longer prompt

It is tempting to treat context engineering as prompt writing with more words. That view misses the main work. A prompt is an instruction given to the model. Context is the full working set the model receives for a task. The prompt is only one part of that set.

Useful context often has several ingredients. There is the user's goal, the role the system should play, the facts it may rely on, the constraints it must obey, examples of good output, records of prior turns, and sometimes live results from tools. Each ingredient can help or harm depending on the task.

More context is not automatically better. Models have a context window, which is the maximum amount of input they can consider at once. Filling it with stale, duplicated, or loosely related material can bury the important facts and raise the chance of a confused answer.

⚠️

Length is not quality. A shorter context with the right evidence usually beats a long context full of noise.

Context is a working set of goal and role, facts, constraints, and examples with history.

The basic pattern: select, shape, send, check

Most context engineering systems follow the same pattern. First they select relevant material. Then they shape it into a form the model can use. Then they send it with the task. Finally they check the result against the goal, the evidence, and the rules.

Selection is the art of deciding what belongs in the working set. If the user asks about a refund policy, the system may need the current policy, the user's order record, regional rules, and the date of purchase. It probably does not need the entire help center.

Shaping is the art of making context legible. A model handles clear labels, short excerpts, explicit dates, stable identifiers, and plain instructions better than it handles a pile of unmarked text. The goal is not to flatter the model. The goal is to reduce ambiguity.

Systems select relevant material, shape it, send it with the task, then check the result.

Grounding: tying answers to evidence

Grounding means connecting a model's answer to specific evidence rather than letting it rely only on general training patterns. In business systems, grounding is often what separates a useful assistant from a confident guesser.

One common method is retrieval-augmented generation, or RAG. Retrieval means searching a knowledge source, such as product documentation or policy records, for relevant passages. Augmented generation means giving those passages to the model so it can draft an answer using them.

Grounding is not the same as truth. A retrieved passage can be outdated, irrelevant, or misread. The system still needs good source management, clear citations where appropriate, freshness checks, and evaluation. Grounding improves the odds; it does not remove the need for judgment.

Prefer authoritative sources over convenient ones.
Attach dates, versions, and ownership to important documents.
Ask the model to distinguish evidence from inference.
When the stakes are high, require a human or system check before action.

Tools turn context into a live workspace

A tool is an external capability the model can use or request, such as a search engine, calculator, calendar, ticketing system, code runner, or database query. Tools matter because some facts should not be memorized and some tasks require action, not just text.

Without tools, a model asked for an account balance may guess, refuse, or explain that it lacks access. With a properly governed tool, the system can fetch the current balance, place the result in context, and have the model explain it to the user. The model is still generating language, but the critical fact came from the system of record.

Tool use adds risk. The model may call the wrong tool, pass the wrong arguments, or treat a tool result as broader than it is. Good context engineering therefore includes tool descriptions, permission boundaries, input validation, output checks, and clear rules for when not to act.

⚠️

Actions need gates. A model that can change records, spend money, or message customers needs stronger controls than a model that only drafts text.

Safe tool use needs descriptions, permission boundaries, input validation, and output checks.

Evaluation is how context becomes engineering

An engineered system is not defined by good intentions. It is defined by behavior that can be observed, tested, and improved. Evaluation is the practice of measuring whether the context design produces the results you need.

A useful evaluation set contains realistic tasks, expected behaviors, and examples of failure. For a support assistant, that might include easy questions, ambiguous questions, missing-account cases, policy edge cases, adversarial requests, and questions whose correct answer is to say it does not know.

The point is not to find one perfect prompt. The point is to learn which context choices improve reliability. Does adding examples help? Does a shorter retrieved passage beat a longer one? Does the model follow the policy if it appears before the evidence instead of after it? These are engineering questions, not matters of taste.

Define the task and the cost of failure.
Build a small set of representative test cases.
Measure answers for correctness, completeness, safety, and usefulness.
Change one part of the context design at a time.
Keep the tests as the product, data, and policies change.

Decisions leaders and practitioners must make

Context engineering quickly becomes an organizational design problem. Someone must decide which sources are trusted, who owns them, how fresh they must be, what the model is allowed to do, and how failures are handled. These choices cannot be delegated entirely to a prompt.

Practitioners face trade-offs every day. A broad retrieval search may find more relevant facts but also bring in noise. A tight policy may reduce risk but frustrate legitimate users. A human review step may improve safety but slow the process. A cheaper model may work if the context is well structured, while a larger model may still fail with poor evidence.

Leaders should ask for a context map before asking for a model roadmap. The map should show the task, the information sources, the tools, the policies, the checks, the owners, and the failure paths. That map reveals whether the system is ready for production more clearly than a demo does.

✅

Best first question. Ask what the model needs to know and prove before asking which model to buy.

Key takeaways

Context engineering designs the full input environment around a model call, not just the prompt.
Good context is selective, structured, current, and tied to the task.
Grounding and tools help models use live evidence, but they also require checks and controls.
Evaluation turns context design from guesswork into an engineering practice.
The hardest choices are often about ownership, risk, freshness, and permission, not model syntax.

A context map shows the task, sources and tools, policies and checks, and owners with failure paths.

Glossary

Context: The full working set of instructions, facts, history, tool results, and constraints given to a model for a task.
Context window: The maximum amount of input a model can consider at one time.
Token: A piece of text, such as a word fragment or punctuation mark, that a language model uses as a basic unit of input and output.
Prompt: An instruction or request given to a model to guide its response.
Large language model: Software trained on large amounts of text to generate language based on patterns in its input and training.
Grounding: The practice of tying a model's answer to specific evidence supplied at the time of the task.
Retrieval-augmented generation: A method that searches external sources for relevant information and gives that information to a model before it answers.
Tool: An external capability, such as a database query or calculator, that a model-driven system can use to get information or take action.
Evaluation: The practice of testing model outputs against defined tasks, expectations, and failure cases.