Converting from Parquet to Delta Lake fails

Problem

You are attempting to convert a Parquet file to a Delta Lake file.

The directory containing the Parquet file contains one or more subdirectories.

The conversion fails with the error message: Expecting 0 partition column(s): [], but found 1 partition column(s): [<column_name>] from parsing the file name: <path_to_the_file_location>;.

Cause

The conversion process is attempting to process the subdirectory as a partition. This causes the error message.

Solution

If you are using Databricks Runtime 7.5 or below, ensure that directories containing Parquet files do not have subdirectories.

This issue is resolved in Databricks Runtime 8.0 and above.