Problem
You are attempting to create a table from a Delta Sharing catalog in your Databricks workspace and you get an error message.
Example error
UncheckedExecutionException: io.delta.sharing.spark.util.UnexpectedHttpStatus: HTTP request failed with status: HTTP/1.1 400 Bad Request
Example error details
"error_code" : "BAD_REQUEST", "message" : "Failed request to sharing server\nEndpoint: https://<workspace-name>.databricks.com:443/api/2.0/delta-sharing/metastores/<metastore-id>/shares/<catalog-name>/schemas/<schema-neme>/tables/menu/metadata\nMethod: GET\nHTTP Code: 400\nStatus Line: 400\nBody: { \"error_code\" : \"BAD_REQUEST\", \"message\" : \"\\nTable property\\ndelta.enableDeletionVectors\\nis found in table version: 1.\\nHere are a couple options to proceed:\\n 1. Use DBR version 14.1(14.2 for CDF and streaming) or higher or delta-sharing-spark with version 3.1 or higher and set option (\\\"responseFormat\\\", \\\"delta\\\") to query the table.\\n 2. Contact your provider to ensure the table is shared with full history.\\n[Trace Id: 58bd2d3f99bdcd2f2a302f977074]\"}\n[Trace Id: 58bd2d3f99bdcd2f2a302f977074]\n"
This issue typically arises in environments utilizing Delta Sharing and a mixture of Databricks Runtime versions.
Cause
The table being accessed has the enableDeletionVectors
property enabled, which is not supported by the Databricks Runtime version you are using. The server returns a 400 Bad Request
error.
For more information, review the What are deletion vectors? (AWS | Azure | GCP) documentation.
Solution
If you are using Delta Sharing, you should upgrade your clients to a supported Databricks Runtime (recommended) or disable the enableDeletionVectors
property on the source Delta table.
Upgrade Databricks Runtime
Info
You should use Databricks Runtime 14.3 LTS or above if you want to use CDF (Change Data Feed) or streaming.
- Stop your cluster.
- Change the Databricks Runtime to 14.1 or above.
- Restart the cluster.
- Create the table.
This approach ensures compatibility with the enableDeletionVectors
property.
Disable enableDeletionVectors
If you cannot upgrade your Databricks Runtime version, you can disable the enableDeletionVectors
property on the source Delta table. This makes the source table readable by Delta Lake clients that do not support deletion vectors.
- Ensure that you have
MODIFY
permissions on the source Delta table. - Access the table from a cluster running Databricks Runtime 14.1 or above.
- Follow the directions in the Drop Delta table features (AWS | Azure | GCP) documentation to disable the
enableDeletionVectors
property. - Run
VACUUM
to delete old files and ensure no concurrent write operations are running. - Access the table from the client.