Problem

While using Databricks apps, you notice you cannot run queries to access files on a Unity Catalog volume. You receive the following error.

FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/<catalog>/<schema>/<path>'

You also notice your queries are successful on general compute. 

Cause

You are attempting to access the volume directly. Databricks apps do not support direct access to Unity Catalog volumes without using the SDK. 

Solution

Download the file from the Unity Catalog volume using the Databricks SDK and then read the file. 

First, install the Databricks SDK if it is not already installed.

%sh
pip install databricks-sdk

Next, use the following code to download the file and read its contents.

%python
from io import BytesIO
import pandas as pd
from databricks_sdk import ServiceClient
# Initialize the Databricks SDK client
client = ServiceClient(token='<your-databricks-token>')
# Download the file from the Unity Catalog volume
response = client.files.download("/Volumes/<path-to-file-in-UC>")
# Read the file contents
file_content = BytesIO(response.contents.read())
df = pd.read_csv(file_content)

For more information, refer to the Develop Databricks Apps (AWS | Azure) documentation. 

Alternatively, you can use the following code to authenticate the service principal using WorkspaceClient() and then download and read the file contents. 

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
volume_contents = w.files.list_directory_contents("/Volumes/<catalog>/<schema>/<volume>/<sub-folder-if-needed>/")
# Obtain files from UC volume into local app memory 
downloaded_files_contents = []
for file in volume_contents:
  if not file.is_directory:
    download_file = w.files.download(file.path)
    downloaded_files_contents.append(download_file.contents.read())

For more information about authentication and w.files, refer to the w.files: Files and Authentication SDK documentation.