“java.lang.IllegalStateException: Unexpected type: JSON” error when creating an external table from BigQuery

Upgrade your cluster to Databricks Runtime 14.0 or above.

Last published at: October 22nd, 2024

Problem

When running the command to create an external table from BigQuery, you encounter the following error.

com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: java.lang.IllegalStateException: Unexpected type: JSON

Your Databricks Runtime version is below 14.0. JSON read support was backported into Databricks Runtime 14.0 and above.

Upgrade your cluster to Databricks Runtime 14.0 or above to create your external table.

For more information on creating external tables with BigQuery please review the Google BigQuery documentation.