SHOW DATABASES command returns unexpected column name

Running the `SHOW DATABASES` command returns an unexpected column name.

Written by Jose Gonzalez

Last published at: May 24th, 2022

Problem

You are using the SHOW DATABASES command and it returns an unexpected column name.

Cause

The column name returned by the SHOW DATABASES command changed in Databricks Runtime 7.0.

  • Databricks Runtime 6.4 Extended Support and below: SHOW DATABASES returns namespace as the column name.
  • Databricks Runtime 7.0 and above: SHOW DATABASES returns databaseName as the column name.

Solution

You can enable legacy column naming by setting the property spark.sql.legacy.keepCommandOutputSchema to false in the cluster’s Spark config (AWS | Azure | GCP).