Problem
While attempting to display sample data or run SELECT * queries on DLT pipeline-created views using a job or cluster in single-user access mode, you encounter an [INSUFFICIENT_PERMISSIONS] Insufficient privileges error message which indicates you do not have USE CATALOG permission on the __databricks_internal catalog.
Example error message
[INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have USE CATALOG on Catalog '__databricks_internal'. SQLSTATE: 42501, data: {'type':'baseError','stackFrames':['org.apache.spark.sql.AnalysisException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges:\nUser does not have USE CATALOG on Catalog '__databricks_internal'.Cause
The owner of a job with a cluster using single-user access mode is referencing a materialized view created by a DLT pipeline with a different owner.
Only the owner of a Databricks SQL materialized view can query the materialized view from a single-user access mode cluster. Otherwise, Databricks SQL materialized views can be queried only from Databricks SQL warehouses, Delta Live Tables, and shared clusters running Databricks Runtime 11.3 and above.
Note
The __databricks_internal catalog is an internal catalog used for DLT pipeline materializations. In general, you should not access these tables directly, which is why users do not have access to it by default.
Solution
Use a cluster in shared access mode for your job, or make sure you own both the materialized view and the job.
Important
Strictly limiting permissions for the __databricks_internal catalog is a design decision. Access to this catalog should only be granted for debugging purposes.
If you need to access data in the __databricks_internal catalog, you should request the necessary permissions from your administrator. As a best practice, only a limited number of use should have USE CATALOG on __databricks_internal permissions. It should not be granted to a user or group by default.