MLflow project fails to access an Apache Hive table

Problem

You have an MLflow project that fails to access a Hive table and returns a Table or view not found error.

pyspark.sql.utils.AnalysisException: "Table or view not found: `default`.`tab1`; line 1 pos 21;\n'Aggregate [unresolvedalias(count(1), None)]\n+- 'UnresolvedRelation `default`.`tab1`\n"
xxxxx ERROR mlflow.cli: === Run (ID 'xxxxx') failed ===

Cause

This happens when the SparkSession object is created inside the MLflow project without Hive support.

Solution

Configure SparkSession with the .enableHiveSupport() option in the session builder. Do this as part of your MLflow project.

val spark = SparkSession.builder.enableHiveSupport().getOrCreate()