Problem

You are trying to create a Parquet table using TIMESTAMP, but you get an error message.

Error in SQL statement: QueryExecutionException: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.UnsupportedOperationException: Parquet does not support timestamp. See HIVE-6384

Example code

%sql

CREATE EXTERNAL TABLE IF NOT EXISTS testTable (
  emp_name STRING,
  joing_datetime TIMESTAMP,
)
PARTITIONED BY
  (date DATE)
STORED AS
  PARQUET
LOCATION
  "/mnt/<path-to-data>/emp.testTable"

Cause

Parquet requires a Hive metastore version of 1.2 or above in order to use TIMESTAMP.

Solution

You must upgrade the Hive metastore client on the cluster.

You can do this by adding the following settings to the cluster’s Spark config (AWS | Azure | GCP).

• Databricks Runtime 6.6 and below

  spark.sql.hive.metastore.version 1.2.1
  spark.sql.hive.metastore.jars builtin

• Databricks Runtime 7.0 and above

  spark.sql.hive.metastore.jars /dbfs <path-to-downloaded-jars>
  spark.sql.hive.metastore.version 1.2.1