Problem
When attempting to read the contents of one notebook from another notebook within the same Databricks workspace, you receive an error message.
Attempt to read file at path: /Workspace/Users/<your-email>/Development/your/filepath
Result: [Errno 2] No such file or directory: '/Workspace/Users/<your-email>/Development/your/filepath'
This issue arises when using the following Python function.
with open(<file-path>, 'r') as file:
Cause
The workspace file system does not support direct file operations using standard Python I/O functions like open()
.
Solution
Use the Databricks SDK to read the contents of a notebook.
First, install the Databricks SDK.
pip install databricks-sdk
Then, use the following code to read the notebook contents. This code uses the Databricks SDK to export the notebook content and decode it from base64 format.
from databricks.sdk import WorkspaceClient
from databricks.sdk.service import workspace
import base64
def notebook_reader(file_location):
databricksURL = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiUrl().getOrElse(None)
myToken = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().getOrElse(None)
w = WorkspaceClient(host=<databricksURL>, token=<yourToken>)
export_response = w.workspace.export(<your_file_location>, format=workspace.ExportFormat.SOURCE)
notebook_content = base64.b64decode(export_response.content).decode('utf-8')
return notebook_content
file_location = ""
notebook_content = notebook_reader(file_location)
print(notebook_content)