SFT: Instruction-Tuning Foundations — Post-Training & Alignment

A large language model does not begin life as a helpful assistant. It begins as a statistical system trained to continue text. The gap between “continue this text” and “answer this user well” is where supervised fine-tuning lives. Supervised fine-tuning, often shortened to SFT, is the practice of teaching a model with examples of instructions and desired responses so that it learns a style of behavior: how to answer, what to include, what to refuse, and what kind of reasoning or format is useful.

Start with the simplest idea: imitation

Supervised fine-tuning is imitation learning for language. You show the model many examples of a user instruction and the response you wish the model had written. During training, the model is rewarded for assigning higher probability to the words in the desired response and lower probability to alternatives.

The word “supervised” means the training examples include the answer, not just the question. The model is not discovering a goal on its own. A human team, a domain process, or another trusted source has supplied examples of the behavior to copy.

This is different from simply writing a better prompt. A prompt guides the model at the moment of use. Supervised fine-tuning changes the model’s learned parameters, so the behavior becomes part of how the model tends to respond across many future prompts.

What the base model already knows

Before supervised fine-tuning, a modern large language model has usually gone through pretraining. In pretraining, the model reads very large collections of text and learns to predict the next token, where a token is a small piece of text such as a word, part of a word, or punctuation.

This process gives the model broad language ability. It learns grammar, facts, patterns of reasoning, code shapes, document styles, and many associations found in the training text. But pretraining alone does not define the behavior you expect from an assistant. It does not reliably know when to be concise, when to ask for clarification, how to follow your organization’s template, or how to avoid answering outside a policy boundary.

SFT is therefore not the stage that creates most of the model’s world knowledge. It is the stage that bends existing capability toward a target behavior. A small, high-quality fine-tuning set can be powerful because it is steering a model that already has broad linguistic and conceptual skill.

📌

Steering, not stuffing. SFT is usually best at teaching behavior, format, and task style, not at loading a large private knowledge base into the model.

What an instruction-tuning example contains

The basic unit of SFT data is an example. Each example contains the context the model should see and the answer it should learn to produce. In a chat model, that context is often represented as messages: a system message that sets durable instructions, a user message that asks for something, and an assistant message that provides the desired answer.

Good examples are specific. They show the exact level of detail, tone, structure, safety boundary, and refusal style you want. If the desired output must be a valid JavaScript Object Notation object, the example should contain valid JavaScript Object Notation. If the answer should cite a source, the example should show what a citation looks like.

Poor examples teach poor behavior. If some examples are verbose and others are terse for no reason, the model learns inconsistency. If labels include hidden shortcuts, outdated policy, or incorrect answers, SFT will faithfully imitate those mistakes.

Use real tasks or realistic synthetic tasks, not toy prompts that hide the hard parts.
Keep the target answer aligned with the behavior you want in production.
Include edge cases, refusals, and clarification questions, not only successful happy paths.
Remove duplicates and near-duplicates so the model does not memorize a narrow pattern.

A chat SFT example is three messages: durable system rules, the user's request, and the desired assistant answer.

How training changes the model

During SFT, the model reads the prompt portion of an example and predicts the response one token at a time. For each next token, the training system compares the model’s probability distribution with the correct token from the target response. The difference is called loss.

Training tries to reduce loss across the examples. It does this by making small updates to the model’s parameters, which are the numerical values that shape how the model transforms input text into output probabilities. After many examples, the model becomes more likely to produce responses that resemble the training targets.

This is why SFT can make a model better at a specialized task without changing the software around it. The model itself has been nudged toward the pattern. But the same fact creates risk: if the examples are biased, unsafe, or too narrow, those patterns can also be learned.

For each token the model predicts, compares to the target, measures loss, and nudges its parameters.

What SFT is good at, and what it is not

SFT is strong when the goal is a repeatable behavior. Examples include answering in a house style, classifying support tickets with a known taxonomy, producing structured extracts, following a domain-specific workflow, or refusing requests according to a written policy.

SFT is weaker when the goal is fresh factual recall over a large or changing body of information. If the model needs current prices, policy documents, product catalogs, or customer-specific records, retrieval is often a better first tool. Retrieval gives the model relevant information at run time instead of trying to bake that information into the model during training.

SFT also does not guarantee truth. It can teach a model to sound like the examples. If the target answers contain confident but unsupported claims, the tuned model may become a more consistent producer of confident unsupported claims.

Use SFT for behavior, style, format, task procedure, and policy-aligned responses.
Use retrieval when answers depend on a large or frequently changing knowledge source.
Use prompt engineering when the behavior can be achieved reliably without changing model parameters.
Use product controls and application logic for hard requirements that must not depend on model judgment alone.

⚠️

Do not tune around missing data. If the model lacks needed facts at inference time, SFT may only teach it to guess in a more polished way.

SFT excels at repeatable behaviors but is weak for fresh facts, changing data, or guaranteeing truth.

How to judge whether tuning worked

A fine-tuned model should be evaluated on examples it did not see during training. This held-out data is commonly called a validation set or test set. Without it, you can mistake memorization for learning.

Evaluation should combine automatic checks and human review. Automatic checks are useful for objective properties such as valid format, exact labels, required fields, and prohibited phrases. Human review is still important for judgment-heavy qualities such as helpfulness, reasoning quality, tone, and safe refusal behavior.

The most common failure is overfitting, where the model becomes very good at the training examples but worse at new examples. Another common failure is regression, where the model improves on the target task but loses useful general behavior from the base model. A disciplined evaluation plan checks both the specialized task and broader assistant quality.

The real decisions for teams

The practical question is rarely “Can we fine-tune?” It is “Should we fine-tune now, and what will we accept as evidence that it helped?” A leader or practitioner must weigh cost, data readiness, governance, latency needs, and the risk of changing model behavior.

Start by writing the behavior you want in plain language. Then test whether prompting, retrieval, tool use, or application logic can meet the requirement. If the behavior remains inconsistent and you can collect high-quality examples, SFT becomes a serious option.

In a Microsoft environment, Microsoft Foundry can help teams manage model experiments, evaluations, and deployment workflows, while identity and access should be governed through Microsoft Entra ID. The important architectural point is not the brand of the platform; it is that training data, evaluation results, model versions, approvals, and rollback plans must be treated as production assets.

Define the target behavior and the unacceptable failures.
Prove the gap with baseline prompts and evaluation data.
Prepare examples that represent real production variety.
Tune a small experiment and compare it with the baseline.
Ship only with monitoring, rollback, and ownership in place.

✅

A useful test. Fine-tune when you can name the behavior, show examples of it, measure it, and operate the resulting model safely.

Key takeaways

Supervised fine-tuning teaches a model to imitate desired instruction-response examples.
SFT mainly steers behavior, format, tone, and workflow; it is not the best way to store large changing knowledge.
Training works by reducing token prediction loss and updating model parameters.
The quality and consistency of the training examples strongly shape the tuned model’s behavior.
Evaluation on held-out data is essential to detect memorization, overfitting, and regressions.
Teams should fine-tune only when the target behavior is clear, measurable, and worth operating as a model change.

Try prompting and retrieval first; only reach for SFT when behavior stays inconsistent and good examples exist.

Glossary

Large language model: A model trained on text to predict and generate language, often capable of tasks such as answering questions, summarizing, and writing code.
Pretraining: The early training stage in which a model learns broad language patterns by predicting text across very large datasets.
Supervised fine-tuning: A training method that adjusts a model using examples that contain both an instruction and the desired response.
Prompt: The input text or messages given to a model at the time it is asked to produce an answer.
Token: A small unit of text, such as a word, part of a word, number, or punctuation mark, that a language model processes.
Loss: A numerical measure of how far the model’s predicted next tokens are from the target tokens in the training data.
Parameters: The numerical values inside a model that are adjusted during training and determine how inputs are turned into outputs.
Validation set: A collection of examples not used for training, used to judge how well the model performs on new cases.
Overfitting: A failure mode where a model performs well on training examples but poorly on new examples because it learned the examples too narrowly.