Scanning Your Data Estate — Purview: Discovery and Catalog

A data estate is unknown until something has looked at it. In Microsoft Purview, that act of looking is not magic and it is not the same as moving the data into Purview. You register a source so Purview knows where it is and how to reach it, then you scan it so Purview can collect metadata, identify likely sensitive information, and make the asset visible in the Data Map and the Unified Catalog.

The basic pattern is register, then scan

Every source follows the same operating pattern. First, you register the data source. That means you tell Microsoft Purview that the source exists and provide a credential or access path that allows Purview to read what it needs to read. Second, you run a scan. The scan connects to the source, reads technical metadata, applies classification rules, and writes the results into the Purview Data Map.

This separation matters. Registering a source is like adding an address book entry; scanning is the visit. A source can be registered and still invisible to catalog users if it has not been scanned successfully. Equally, changing a scan rule set, credential, or schedule does not change the source itself; it changes how Purview inspects it.

📌

Simple model. Registering says the source exists; scanning makes its metadata and classifications available to Purview.

Every source is registered, then scanned, classified, and written to the Data Map.

Different source types, one mental model

The connector changes by platform, but the pattern does not. An on-premises SQL Server database behind a corporate firewall is usually scanned through an integration runtime, which is small software installed inside the network. The integration runtime allows Purview to reach the database securely without requiring the database to be exposed directly to the public internet.

Azure-native sources, such as Azure SQL Database, Azure Data Lake Storage, Azure Synapse Analytics, and Azure Cosmos DB, are usually simpler to connect because Purview and those services run within the same cloud environment. The work is still deliberate: choose the source type, configure the credential or managed identity, define the scope, and run or schedule a scan.

Non-Azure sources follow the same high-level pattern through specific connectors. Amazon Web Services (AWS), Google Cloud, and software as a service applications each have their own details, permissions, and limits, but they still feed the same Purview Data Map when scanning succeeds.

Microsoft Fabric OneLake is not automatic

Microsoft Fabric can create a false sense of coverage because it is part of the Microsoft analytics ecosystem. Its underlying storage layer, OneLake, is not automatically included in the Purview Data Map merely because an organization uses Fabric. It must be explicitly registered and scanned like other sources.

This is a common blind spot. A company may scan Azure storage accounts, databases, and warehouses carefully, while a fast-growing Fabric environment remains outside the governed view. If analysts are using OneLake heavily for lakehouses, warehouses, shortcuts, or shared analytics work, leaving it unscanned means the catalog is missing a material part of the estate.

The practical check is simple: do not ask whether the organization uses Fabric; ask whether OneLake has been registered as a Purview source and whether recent scans have completed successfully.

⚠️

Easy to miss. OneLake must be registered and scanned explicitly; Fabric usage alone does not make it appear in the Data Map.

OneLake is only governed once it is explicitly registered and scanned.

How an Amazon S3 scan works

Amazon Simple Storage Service (Amazon S3) is a useful example because it shows the multicloud pattern clearly. In Microsoft Purview, an administrator adds the AWS account as a data source and uses a credential based on an AWS role Amazon Resource Name (ARN). That role grants Purview secure, read-only access to the selected account resources.

The administrator then selects one or more S3 buckets, chooses a scan rule set, and decides whether the scan runs once or on a recurring schedule. A scan rule set defines practical boundaries: which file types are in scope and which classification rules should be applied. Purview includes a built-in Amazon S3 default rule set, and teams can also create custom ones.

During the scan, the connector reads object metadata and inspects content for classification. Purview includes more than two hundred built-in detectors for sensitive information, such as credit card numbers, Social Security numbers, national insurance numbers, and application programming interface (API) keys. The important privacy point is that Purview sends metadata and classification results back to Purview; it does not copy the underlying sensitive data into Purview.

What a scan actually produces

A successful scan does not produce a copy of the source system. It produces a governed description of the source. For structured databases, that can include schema detail such as databases, tables, columns, and data types. For file stores, it can include file names, folder paths, object metadata, file sizes, and detected file types. Across both, it can include classifications and, where configured, sensitivity label information.

Those results become useful in three main places. The Data Map shows the registered asset and its scan history, which helps administrators understand when the source was last inspected and whether the scan succeeded. Insight Reports summarize patterns across the estate, such as classifications, sensitivity labels, and file types. The Unified Catalog makes scanned assets searchable so business and technical users can find data across Azure, Fabric, on-premises systems, and other clouds through one catalog experience.

This is why scanning is foundational to governance. Policies, stewardship, lineage discussions, and data quality work all start from the same question: what do we have, where is it, and what does it appear to contain?

A successful scan surfaces results in three governed places.

The operating decisions behind scanning

Scanning is not only a technical setup task. It forces decisions about scope, risk, cost, timing, and accountability. A leader wants confidence that important data is visible without creating unnecessary operational noise. A practitioner wants scans that complete reliably, use the right permissions, and produce classifications that people trust.

The first decision is scope. Scanning everything at maximum depth may sound attractive, but it can be slow and expensive in complex estates. A better starting point is often to prioritize regulated, customer, financial, employee, and operationally critical data stores, then expand coverage in waves.

The second decision is cadence. Some sources change hourly; others change quarterly. Recurring scans should match the rate of meaningful change and the needs of downstream processes such as catalog search, sensitivity review, compliance reporting, and data stewardship.

The third decision is ownership. Someone must know who owns the credential, who approves the scan scope, who responds when a scan fails, and who decides whether a classification result is correct. Without those roles, a scanning program becomes a one-time inventory rather than an operating discipline.

Prioritize high-risk and high-value sources before trying to scan everything.
Use the least privilege needed for scanning, typically read-only access for metadata and classification.
Set scan schedules according to how often the source changes and how quickly users need updated catalog results.
Assign owners for credentials, source scope, scan failures, and classification review.
Explain to stakeholders that scanning improves visibility; it does not by itself fix data quality, access control, or retention.

Scanning forces two operating decisions: what to scan and how often.

Timing and failure modes to plan for

A scan is not always instant. Once started, a scan can take up to twenty-four hours to complete before results and Insight Reports are fully available. This is especially important during demonstrations, workshops, and first-time onboarding. A newly registered source may not appear in search five minutes after someone clicks the scan button.

Failed scans are usually mundane rather than mysterious. Common causes include missing permissions, expired credentials, network connectivity problems, incorrect source scope, unsupported file types, and policies that block required operations. For Azure storage accounts, organizational Azure Policies that block storage account updates can cause Purview scans to fail. Microsoft provides a dedicated policy exclusion for Purview scanning accounts, so checking policy effects is a useful early diagnostic step.

The healthiest scanning programs treat scan history as operational telemetry. They review failures, fix root causes, and use the results to improve onboarding patterns. The goal is not just to run scans, but to keep the estate discoverable as it changes.

⚠️

Set expectations. Do not promise immediate catalog results; a scan can take up to twenty-four hours to complete.

Key takeaways

Every source follows the same pattern: register it, then scan it.
Registration gives Purview a location and access path; scanning collects metadata and applies classification rules.
On-premises sources often use an integration runtime, while Azure and multicloud sources use their own connectors and permission models.
Microsoft Fabric OneLake is not automatically included in the Data Map and must be registered and scanned explicitly.
The Amazon S3 connector can inspect metadata and content for classification, but it returns metadata and classification results rather than copying sensitive source data into Purview.
Plan for scan duration, permissions, policy exceptions, ownership, and recurring failure review.

Glossary

Register: To tell Microsoft Purview that a data source exists and provide the access details needed to scan it.
Scan: The process by which Purview connects to a source, reads metadata, applies classification rules, and records the results.
Data Map: The Purview metadata store that records scanned assets, their relationships, classifications, and scan history.
Unified Catalog: The Purview search and discovery experience where users find governed data assets from scanned sources.
Integration runtime: Software installed inside a network to let Purview securely scan on-premises sources without exposing them directly to the internet.
OneLake: The storage layer for Microsoft Fabric that must be explicitly registered and scanned before it appears in Purview governance views.
Scan rule set: A configuration that defines which file types and classification rules a scan applies to a source.
Amazon Resource Name: An Amazon Web Services identifier, often abbreviated as ARN, that uniquely names a cloud resource such as an access role.
Insight Reports: Purview reports that summarize scan results such as classifications, sensitivity labels, and file types across the estate.