Shared table not accessible in Delta Sharing using Python

Make sure correct access permissions are set and no network restrictions exist.

Written by amrith.v

Last published at: September 12th, 2024

Problem

You receive a FileNotFoundError pointing to an Azure storage URL when trying to access a shared table in Delta Sharing using Python. 

Example

import delta_sharing
profile_file = "<profile-file-path>"
client = delta_sharing.SharingClient(profile_file)
# List all shared tables.
client.list_all_tables()
# Create a url to access a shared table.
table_url = profile_file + "#<share-name>.<schema-name>.<table-name>"
# Load a table as a Pandas DataFrame.
delta_sharing.load_as_pandas(table_url)

Error message: 

FileNotFoundError: https://<Storage account>.dfs.core.windows.net/<table details>/part-00000.snappy.parquet?<request details>

Cause

If the Azure Blob Storage has network restrictions, it may prevent the Databricks service from accessing the required data. 

Additionally, the storage principal or managed identity used may not have the required access permissions to the storage account. The STORAGE BLOB DATA CONTRIBUTOR role may not be assigned at the storage account level, or there may be a need to set the STORAGE BLOB DELEGATOR role at the storage account level and provide the STORAGE BLOB DATA CONTRIBUTOR role at the container level.

Solution

  1. Ensure that the storage principal or managed identity used has the required access to the storage account. Assign the STORAGE BLOB DATA CONTRIBUTOR role at the storage account level.
  2. If it is not possible to provide the STORAGE BLOB DATA CONTRIBUTOR role at the storage account level, set the STORAGE BLOB DELEGATOR role at the storage account level and provide the STORAGE BLOB DATA CONTRIBUTOR role at the container level.
  3. Verify that there are no network restrictions on the Azure Blob Storage that could prevent the Databricks service from accessing the required data.
  4. Test the access to the shared table again after making the necessary changes to the access permissions and network settings.

 

For more information on how to grant the managed identity access to the storage account, please refer to the Use Azure managed identities in Unity Catalog to access storage documentation.