Members not supported SCIM provisioning failure

You get a members or groups not supported error when trying to provision new users to your workspace via SCIM....

Cannot delete Unity Catalog metastore using Terraform

Set force_destroy in the databricks_metastore section of the Terraform configuration to delete a metastore and its catalog....

Permission denied error when creating external location

You must be a metastore admin or have the CREATE EXTERNAL LOCATION privilege....

Overlapping paths error when querying both Hive and Unity Catalog tables

If a Hive table and a Unity Catalog table both refer to the same external storage path you cannot query them in the same notebook cell....

Failed to create query error when upgrading external metastore to Unity Catalog

The "Create query for upgrade" command only works when run on a warehouse in Data Explorer....

SYNC command fails with a mismatched input error

Unity Catalog must be enabled on your cluster before you can use SYNC to migrate a legacy Hive table to Unity Catalog....

Unable to access Unity Catalog views

You cannot access a view if you do not have sufficient permissions on the underlying table....

INVALID_PARAMETER_VALUE.LOCATION_OVERLAP: overlaps with managed storage error

External tables cannot overlap with catalog/schema storage locations. They should be created in a subdirectory instead....

Cluster-scoped init script to Unity Catalog volume migration notebook

Easily migrate your cluster-scoped init scripts to Unity Catalog volumes....

List all available tables and their source formats in Unity Catalog

Use system.information_schema.tables to display available table names and their data source formats....

No way to restore dropped managed volumes

To reduce risk, always use external volumes for operations that require frequent schema changes. ...

Unsupported path error when creating an external table

Create the table in a different location that does not overlap with the external volume. ...

Azure Synapse Analytics (formerly SQL Data Warehouse) federated queries throwing "Invalid object name 'dbo.date'" error

Set the collation for the database in the dedicated SQL pool to "SQL_Latin1_General_CP1_CI_AS". ...

You get an Insufficient privileges on __databricks_internal catalog error when attempting to query DLT pipeline views

You should not access the __databricks_internal catalog directly. It is an internal catalog used for DLT pipeline materializations....

Failure to initialize configuration for storage account while registering a model serving model in Unity Catalog

An external location must be provisioned during the initial Unity Catalog configuration....

Find your metastore ID

Step-by-step instructions to find the metastore ID....

Access denied error when trying to read or write to an S3 bucket in a separate AWS account

Disable ACLs in the BucketOwnerEnforced setting in S3 Object Ownership, or ensure that access to the objects in the S3 bucket is permitted....

Unable to view, delete, or drop an external location in the UI or through commands, even with admin privileges

Use metastore admin rights to change ownership to yourself or the required user, and then manage. ...

Error [DELTA_UNSUPPORTED_TIME_TRAVEL_MULTIPLE_FORMATS] while creating VIEW

Use a three-level namespace to CREATE VIEW based on UC table version. ...

Circular reference error when trying to apply row filter function in Databricks SQL analytics

Remove the circular reference in the row filter configuration....

Comments not reflecting on Unity Catalog tables

Update the Apache Spark configuration to enable automatic metadata updates. ...

Job failing when trying to write dataset to external path

Clarify which table is using which location on a selected schema. ...

Bad Request error when creating a table from a shared catalog

Upgrade the client’s Databricks Runtime version or disable the enableDeletionVectors property on the source table....

Need to track DBU consumption per cluster and present clusters in the workspace

Use system tables to query DBU consumption....

Delta external table metadata does not match the Catalog explorer view

Run the MSCK REPAIR command. ...

How to find Delta table name based on table id

Use system.information_schema.tables to identify the Delta table name from table id....

Model lineage not showing source Delta tables in the graph for Databricks Runtime 15.3 or above

Load the data using MLflow load_delta format and log the input....

Running queries using Hive Federation connection throwing SparkException error

Disable ACL to update the S3 bucket owner permission type. ...

"Duplicate tag keys are not allowed" when updating table tag keys

All tag keys must be lower case after 19th November 2024....

Unable to access secrets using instance profile in shared access mode

Use a single-user access mode cluster instead....

“Unresolvable Table Valued Function” error when trying to execute CREATE OR REPLACE on a table with read and write permissions

Ask the table owner to transfer ownership to you or execute CREATE or REPLACE commands for you....

IAM role access denied on cross-account S3 buckets in Unity Catalog

Ensure that the IAM roles trust and access policies are correctly configured....

NucleusUserException error when trying to create an SQL warehouse cluster

Check global init script and manually change the my.cnf file....

Error [JVM_ATTRIBUTE_NOT_SUPPORTED] when trying to obtain the number of partitions in a Dataframe

Switch to single-user cluster or use the spark_partition_id() function in a shared cluster....

Using Terraform to rename a metastore in Unity Catalog destroys and recreates it instead

Use the REST API or the Unity Catalog CLI to rename the metastore....

Error creating tables on foreign catalogs in Databricks Lakehouse Federation

You should create tables directly in the external data source and access them through Unity Catalog....

PyJError when using the toJson method in standard access mode compute

Change the method from toJson to safeToJson....

Insufficient permission error when querying views in dedicated access mode

Grant necessary privileges, use alternative compute resources, or use a newer Databricks Runtime....

Error when trying to load a dataset after integrating Unity Catalog metadata with Power BI

Increase the byte limit in your ODBC settings and use the native query option in Power BI....

Permission denied error when trying to run VACUUM command on Unity Catalog table with dedicated compute (formerly single-user cluster)

Use shared compute instead, or ensure proper access permissions to both the source and cloned tables. ...

Error listDatabases() method is not allowlisted when running CatalogImpl.listDatabases() on a no isolation shared cluster

Update to Databricks Runtime 14.1 or above....

Not found 404 errors when selecting system tables

Update DNS resolution to reach the current and correct storage endpoint....

Unable to access Unity Catalog tables using Power BI desktop or Tableau

Use UC-enabled clusters with supported access modes, update your Databricks Runtime version, and check ODBC Driver configuration....

AWS S3 storage credential validation failure due to external ID

The user type that creates the storage credential affects how the external ID is created....

Trying to write Excel files to a Unity Catalog volume fails

Write the Excel files to local storage first and then copy the file to a volume location....

Unable to read/write feather file using Unity Catalog (UC) cluster

Install the PyArrow library, use read_feather() to read the file, and enable Arrow-based columnar data transfers....

List users who executed SELECT command against a list of schemas using system access table

Query the audit logs in the system.access.audit table....

Cannot see how to query the number of users per workspace in SQL Analytics

Use system tables to find the user count....

Metastore admin group was accidentally deleted, no access to the owned catalogs or underlying data

Sync the group from your SCIM provider; otherwise recreate the group and assign necessary permissions....

Cannot access a directory from an external volume in S3 using the Databricks UI

Identify and remove the conflicting file with the same name as the directory. ...

Queries to access files on a Unity Catalog volume fail with FileNotFound error

Download the file from the Unity Catalog volume using the Databricks SDK and then read the file....

SQL compilation error when trying to access a Snowflake-defined view

Confirm the view definition and column discrepancy, modify the view in Snowflake, then synchronize the view definition....

AccessDenied error on Unity Catalog when querying external locations

Update the S3 bucket policy references to the deleted role’s Amazon Resource Name (ARN)....

Error “Cannot find catalog plugin class for catalog” when using a custom catalog plugin in JDBC driver

Place the custom JAR file in /databricks/jars and /databricks/hive_metastore_jars using an init script. ...

Error trying to access a Unity Catalog (UC) volume path from a non-UC compute

Use a UC-enabled compute to access volumes in Databricks....

JVM-based workloads unable to access Unity Catalog volumes

Depending on your workload, use init scripts or configure libraries to use supported storage mechanisms....

How to bulk assign permissions to catalogs

Use a script to store all the catalog names in a Python list and then give a specific user access....

How to get details for an SQL query run in a notebook on an all-purpose cluster

Enable verbose audit logs, then check the system.access.audit table. ...

How to find the last accessed details for a table

Query the system.access.audit table....

How to implement strict access restrictions on system tables

Create a view and grant access to the relevant users....

Databricks-based queries using Snowflake federated tables running slowly

Set spark.databricks.optimizer.aggregatePushdown.enabled to false....

last_altered column in information_schema not reflecting data modifications

Use the DESCRIBE DETAIL command. ...

Not able to see all rows from system.information_schema tables

Assign BROWSE privileges on objects to view data in information_schema....

How to list users who executed SELECT command against a list of schemas using system access table

Use audit logs and add a desired date range....