MLflow project fails to access an Apache Hive table

Resolve "Table or view not found" error when an MLflow project fails to access an Apache Hive table.

Written by vikas.yadav

Last published at: May 16th, 2022

Problem

You have an MLflow project that fails to access a Hive table and returns a Table or view not found error.

pyspark.sql.utils.AnalysisException: "Table or view not found: `default`.`tab1`; line 1 pos 21;\n'Aggregate [unresolvedalias(count(1), None)]\n+- 'UnresolvedRelation `default`.`tab1`\n"
xxxxx ERROR mlflow.cli: === Run (ID 'xxxxx') failed ===

Cause

This happens when the SparkSession object is created inside the MLflow project without Hive support.

Solution

Configure SparkSession with the .enableHiveSupport() option in the session builder. Do this as part of your MLflow project.

%scala

val spark = SparkSession.builder.enableHiveSupport().getOrCreate()