Problem

You run commands like DESCRIBE DETAIL, SHOW CREATE TABLE, SHOW TABLEPROPERTIES to inspect a table schema or properties, or attempt to view the table metadata in the UI using the Data tab. You notice that sometimes, column names or property values are redacted (masked). 

For example, a column named hourly might appear as ****** instead of listing the actual name. The masked information makes it difficult to verify the table structure or debug issues. 

Cause

This masking occurs because of the spark.redaction.regex Apache Spark configuration. This setting redacts values matching certain keywords, such as: 

• access.key
• credential
• oauth2
• password
• secret
• token
• url
• username

The hourly column is redacted because the word “hourly” contains the substring “url”. The redaction regex treats it as matching a risky keyword, causing it to be masked. 

Solution

To prevent unnecessary redaction of column names or table properties, set the Spark config spark.databricks.behaviorChange.SC102534CommandRedactProperties.enabled to false. This setting controls automatic masking of sensitive property values in table metadata commands. 

Run the following command in a notebook before inspecting the table.

spark.conf.set("spark.databricks.behaviorChange.SC102534CommandRedactProperties.enabled", False)