Problem
You see the following system-generated query in your query history page and want to understand where it comes from.
-- This is a system generated query from sql editor
describe table extended `hive_metastore`.`<schema-name>`.`<table-name>`
Cause
The following two scenarios are the most common triggers for system generated queries.
- DBSQL Intellisense and Catalog Explorer automatically trigger when working with tables in Hive metastore (HMS). The queries are part of how Databricks provides features like autocomplete, error highlighting, and schema previews.
- When your workspace default catalog is set to
hive_metastore
, these interactions are more frequent.
- When your workspace default catalog is set to
- The current catalog/schema is set at the query editor level (either USQLE, legacy SQLE or through a USE statement in the query) which can differ from the default catalog set in the workspace.
Solution
System generated queries are part of normal Databricks behavior and are safe. But if they cause concern (such as query log noise or rarely, slowness), they can be optionally disabled or minimized through settings, or by switching to Unity Catalog.
Disable or minimize through settings
To disable Autocomplete on the SQL editor:
- Click the kebab menu in the upper right side of the SQL editor pane.
- Click Disable Autocomplete.
Then navigate to your workspace settings. From the settings landing page:
- Go to User > Developer > Code Editor.
- Toggle off SQL Syntax Error Highlighting to disable error highlighting.
- Toggle off Autocomplete as you type to disable autocomplete.
Switch to Unity Catalog
Switching your default catalog to Unity Catalog (UC) reduces system-generated metadata queries while still allowing full access to HMS tables using a full path, such as hive_metastore.db.table
.
Navigate to your workspace settings.
- Navigate to Workspace Admin > Advanced > Other.
- In the Default catalog text field, type in the workspace level default catalog you want to use.
- Click the Save button.
Then verify your query editor level catalog and schema matches the workspace.
- Navigate to SQL editor in the sidebar.
- Click the
<catalog>.<schema>
name to the right of the Run button to expand the list of options. - Make sure what you have selected matches the catalog and schema you just changed to in the Workspace admin settings.