Govern Unity Catalog Objects

A catalog with hundreds of tables is only useful if people can find the right one, and the cheapest way to make a table findable is to describe it. In Unity Catalog you attach a plain-language description to any securable object with a comment, and that text becomes part of the object's metadata rather than a note in a separate wiki.

Set a table's description with the COMMENT ON^[1] command or the COMMENT clause of CREATE/ALTER TABLE; set a column's description with ALTER TABLE ... ALTER COLUMN ... COMMENT, which is the only way to document an individual column (Add comments to data and AI assets^[2]). You can also edit any comment in Catalog Explorer, where saving one runs an ALTER statement under the covers. A table comment documents what the dataset is; per-column comments document each field.

Because a comment is stored as Unity Catalog metadata, it survives schema changes and shows up wherever the catalog is searched. Any user with the BROWSE privilege can read it, and the text feeds Catalog Explorer search and the natural-language answers in AI/BI Genie (Add comments to data and AI assets^[2]). You write the description once and every discovery surface reuses it, so a user understands a table without opening the files behind it.

For a large or legacy catalog, writing every description by hand is the bottleneck. Catalog Explorer can propose AI-generated comments (also called AI-generated documentation) for a table and for each of its columns; a data steward reviews the suggestion and clicks Accept, or edits it first (Add AI-generated comments to Unity Catalog objects^[3]). The review step is the point: the model proposes and a human approves, so a big catalog gets documented quickly without publishing text nobody checked.

Describe a table and one of its columns

This adds a dataset-level description and a column description; the rest of the schema is unchanged:

COMMENT ON TABLE sales.orders IS 'One row per confirmed customer order';
ALTER TABLE sales.orders ALTER COLUMN email COMMENT 'Customer contact email';
-- ...

COMMENT ON TABLE sets the table description, and the ALTER COLUMN ... COMMENT form is what documents the email column specifically.

Two Unity Catalog controls limit what a principal sees inside a table at query time, and the exam turns on telling them apart: a row filter removes whole rows, and a column mask rewrites individual values. Both are SQL user-defined functions (UDFs) that Unity Catalog evaluates on every query, and both attach to the table itself, so a query through any SQL warehouse, notebook, or client sees the same enforcement (Row filters and column masks^[4]).

A row filter is a boolean SQL UDF bound to a table with ALTER TABLE <table> SET ROW FILTER. Unity Catalog runs the function once per row and keeps the row only when it returns TRUE; a row where the function returns FALSE is excluded from the result (Row filters and column masks^[4]). Use it for row-level security, such as limiting each analyst to their own region's records. What a row filter cannot do is show part of a value: it decides whole rows in or out, with nothing in between.

A column mask is a SQL UDF bound to one column with ALTER TABLE <table> ALTER COLUMN <col> SET MASK. It takes the column value and returns either the original or a redacted version, so every row is still returned but the sensitive column is rewritten, for example revealing only the last four digits of a card or only the domain after the @ in an email (Row filters and column masks^[4]). Reach for a column mask exactly when a row filter is too blunt, when the row must stay visible but one field must be partly hidden.

That distinction drives a least-privilege point worth internalizing. Masking a sensitive column while leaving table SELECT in place lets the query run and return every row, with only the protected field redacted. Revoking access instead is coarser and breaks any query that references the column, so a masked column follows least privilege where a revoke does not.

One hard boundary decides many questions: row filters and column masks attach to base tables (and, through ABAC policies — attribute-based access control rules introduced in the next section — to materialized views and streaming tables), but you cannot place them on a standard view (Row filters and column masks, Limitations^[4]). If a stem puts SET MASK on a CREATE VIEW, it is wrong; to filter or mask through a view, use a dynamic view^[5] whose logic lives in the view definition.

The order of enforcement is easiest to see as a pipeline: the base table's rows enter, the row filter drops the ones its UDF rejects, the column mask rewrites the protected values in the rows that remain, and the caller receives a filtered, redacted result.

Configuring SET ROW FILTER or SET MASK on every table stops scaling the moment you have hundreds of them. Attribute-based access control (ABAC) solves that by protecting data based on a tag instead of a table name: you tag the sensitive data once, write one policy, and it covers every table that carries the tag, today and in the future (Attribute-based access control in Unity Catalog^[6]).

The vocabulary ABAC evaluates is the governed tag: an account-level tag key with a defined set of allowed values, plus permissions that control who may assign it (Governed tags^[7]). Governed tags are enforced across every workspace in the account, so a pii or classification tag means the same thing everywhere. An account can hold up to 1,000 governed tags, each with up to 50 allowed values (Governed tags^[7]).

An ABAC policy is the enforcement half. You attach a policy at a catalog, schema, or table, and its condition targets a governed tag; when an object carries the matching tag, the policy applies its row filter or column mask automatically, to every current and future object that matches (Attribute-based access control in Unity Catalog^[6]). This is the contrast the exam probes: a table-level SET MASK protects one table and must be repeated on the next, while one ABAC policy on a pii tag governs an entire catalog at once. ABAC also supports materialized views and streaming tables, which the table-level form does not (ABAC vs table-level controls^[8]).

Tagging scales because tags flow downhill. A governed tag applied to a catalog or schema is inherited by the objects within it, so tagging a schema propagates the attribute to its tables without touching each one (Governed tags, Inheritance^[7]). One exception is easy to misread and worth pinning down: inheritance reaches the tables inside a catalog or schema but not individual table columns, so a column-level classification has to be tagged on the column directly. Put together, you tag a catalog, attach one policy, and both the tag and its protection reach every descendant table automatically.

The picture is a hierarchy: the tag sits at the top, inheritance carries it down to the tables, and the policy attached at the catalog applies to every object the tag reaches.

Governance is not only about restricting access; it is about answering "where did this data come from" and "who touched it." Unity Catalog answers both automatically, so you query the record instead of building one.

Data lineage is captured automatically for queries and workflows run on Azure Databricks, down to the column level, with no configuration, and it is aggregated across every workspace attached to the metastore (Lineage in Unity Catalog^[9]). In Catalog Explorer, the Lineage tab renders an interactive graph of upstream and downstream tables, columns, notebooks, jobs, and dashboards, alongside the object's owner and history. Before you change or drop a column, that graph is the impact analysis: it shows exactly which downstream tables and dashboards depend on it.

Audit logs record access rather than structure. Every action against a securable is written to the system.access.audit^[10] system table, which captures the principal (user_identity), the securable, and the action taken (action_name, for example getTable), and holds both account-level and workspace-level events. Its free retention period is 365 days (System tables reference^[11]), so one query answers "who read this table last month" across the account.

For retention beyond a year, or to route events into an existing monitoring stack, configure Azure diagnostic settings on the workspace. Diagnostic settings stream the same Azure Databricks audit (diagnostic) logs to a Log Analytics workspace, an Azure storage account, or an Azure Event Hubs namespace, which is how you get long-term archival, Azure Monitor querying, and alerting (Configure diagnostic log delivery^[12]).

By default the audit log records that an operation happened but not the text of what ran. Enabling verbose audit logging adds fine-grained command events, the notebook and jobs runCommand actions and the Databricks SQL commandSubmit and commandFinish actions, which capture notebook-cell and SQL-warehouse command activity that standard logging omits (Enable verbose audit logs^[13]). Turn it on when an audit needs the actual queries, not just the fact that a query ran.

Two destinations for one event are worth picturing: every audit event lands in the queryable system table, and, if you configure it, the same event also streams out through diagnostic settings to one of three Azure sinks.

Delta time travel lets you query or restore an earlier version of a table, but it only reaches as far back as two retention windows allow, and running VACUUM can close that window for good. Two table properties set the windows, and confusing them is a classic trap.

delta.logRetentionDuration controls how long the transaction-log history is kept, and it defaults to 30 days (Table properties reference^[14]). Because time travel resolves a version or timestamp against that log, this property bounds how far back a time-travel query can reference. delta.deletedFileRetentionDuration controls how long data files that have been logically removed are kept before VACUUM may delete them, and it defaults to 7 days (interval 1 week) (Table properties reference^[14]). The two govern different things: one keeps the log (how far back you can name a version), the other keeps the files (whether that version's data still exists on storage).

VACUUM is where recoverability meets cost. It permanently removes data files that are no longer referenced by the latest table version and are older than the retention threshold, which reclaims storage and ensures deleted records are truly gone (Remove unused data files with vacuum^[15]). The trade-off is stated plainly in the docs: the ability to query table versions older than the retention period is lost after running VACUUM (Remove unused data files with vacuum^[15]). Once the files for a version are purged you can no longer time-travel to it, even if a log entry still names it.

The practical rule follows directly. To preserve the ability to roll back or audit older versions, raise delta.deletedFileRetentionDuration before you run VACUUM, and keep delta.logRetentionDuration at least as long as the history you want to name. To reclaim storage or guarantee a deleted record is unrecoverable, run VACUUM and accept that the older versions go with it.

Lengthen the recoverability window before vacuuming

This raises both retention windows so older versions stay recoverable; the values shown are examples, not the defaults:

ALTER TABLE sales.orders SET TBLPROPERTIES (
  'delta.deletedFileRetentionDuration' = 'interval 30 days',
  'delta.logRetentionDuration'         = 'interval 30 days'
);
-- ...

delta.deletedFileRetentionDuration is the window VACUUM honors before it may purge files, and delta.logRetentionDuration bounds the versions time travel can name.

When you give another team or company read access to specific tables, the goal is to expose exactly those tables and nothing else. OpenSharing (previously Delta Sharing) does that with a share: a read-only collection that holds only the tables or views you add, granted to a named recipient (What is OpenSharing?^[16]). The recipient sees the objects in the share and never the rest of the metastore, and you can add, remove, or revoke access at any time.

How the recipient authenticates depends on who they are, and this is the distinction the exam tests. If the recipient works in another Unity Catalog-enabled Databricks workspace, use Databricks-to-Databricks sharing: you create a recipient of authentication type DATABRICKS, matched to the recipient's metastore by a sharing identifier (a cloud:region:uuid string) (Create data recipients for OpenSharing^[17]). No bearer token is created or managed, and identity verification, authentication, and auditing are handled by the platform over a secure Databricks-managed channel. That is the whole advantage: nothing to rotate.

If the recipient is on any other platform, or has no Unity Catalog-enabled workspace, use open sharing (the Databricks-to-Open protocol) with a recipient of authentication type TOKEN. Azure Databricks generates a long-lived bearer token, a credential file that includes the token, and an activation link that you send over a secure channel; the recipient downloads the credential and uses it to authenticate (Create a recipient using bearer tokens^[18]). Because that token grants read access, it must be kept secret and rotated if it is ever exposed. Only this open-sharing path issues a token; Databricks-to-Databricks does not, which is why "eliminate token management" always points to the Databricks-to-Databricks answer.

One governance interaction is worth flagging: a provider cannot share a table that has table-level row filters or column masks applied (What is OpenSharing?, Limitations^[16]). To share a redacted slice, expose a dynamic view rather than the protected base table.

Most govern-objects questions hand you a goal and a constraint, then ask for the single control that satisfies both. Read the stem for the signal and match it.

• Hide whole rows by identity (region, department, owner): attach a row filter with ALTER TABLE ... SET ROW FILTER. If the same rule must cover many tables by classification, use an ABAC policy on a governed tag instead.
• Reveal only part of a value (last four digits, email domain): a column mask with ALTER TABLE ... ALTER COLUMN ... SET MASK. A row filter cannot do partial redaction.
• Let the query keep running while a sensitive column is hidden: mask the column and keep table SELECT; do not revoke column access, which errors the query.
• Apply one rule across a whole catalog, including tables added later: a governed tag plus an ABAC policy at the catalog or schema, not per-table configuration.
• Put a filter or mask on a view: not allowed on a standard view; the controls attach to tables, or you use a dynamic view.
• Find out who read a table: query system.access.audit (365-day retention). To keep logs longer or to alert, stream them via Azure diagnostic settings to Log Analytics.
• Capture the actual command text that ran: enable verbose audit logging (runCommand, commandSubmit, commandFinish).
• Trace what a column feeds before you drop it: the Catalog Explorer Lineage tab, which captures column-level lineage automatically.
• Keep the ability to roll back older versions: raise delta.deletedFileRetentionDuration before VACUUM; once VACUUM purges the files, time travel to those versions is gone.
• Share tables with another Databricks org without managing tokens: OpenSharing Databricks-to-Databricks (recipient type DATABRICKS). For a non-Databricks partner, open sharing issues a bearer token you must secure and rotate.