You may not always know the type of table you need to read. For example, if a given table is a Delta table you may need to read it differently than if it were a Parquet table.
This article explains how you can use Python code in a Databricks notebook to programmatically determine if a table is a Delta table or not.
Instructions
- Attach your notebook to an all-purpose cluster.
- Copy the example code to your notebook.
- Replace the following values in the example code:
- <table-name-to-check> - The name of the table you want to read
- Run the cell.
If the table is a Delta table, the example code returns Yes, it is a Delta table.
If the table is not a Delta table, the example code returns No, it is not a Delta table.
You can use this example code as a basis to build an automatic check into your notebook code.
Example code
%python def delta_check(TableName: str) -> bool: desc_table = spark.sql(f"describe formatted {TableName}").collect() location = [i[1] for i in desc_table if i[0] == 'Location'][0] try: dir_check = dbutils.fs.ls(f"{location}/_delta_log") is_delta = True except Exception as e: is_delta = False return is_delta res = delta_check("<table-name-to-check>") if (res="True") print("Yes, it is a Delta table.") else print("No, it is not a Delta table.")