On Dec 1, 2023 init scripts stored on DBFS (including legacy global init scripts and cluster-named init scripts) reached End-of-Life.
Databricks recommends you migrate any init scripts stored on DBFS to a supported type as soon as possible.
Databricks Engineering has created a notebook to help you detect init scripts stored on DBFS across clusters, cluster policies, jobs, DLT pipelines in your workspace.
You must be a Databricks admin to run this notebook.
Before running this notebook, you should complete the following:
- Have a cluster running Databricks Runtime 13.3 LTS.
- Import the helpers.zip package into your workspace.
- Verify that the /helpers folder exists and contains the files
By default this notebook checks the current workspace. If you want to check additional workspaces, you will need the following:
Detect init scripts stored on DBFS
- Download the DBFS init script detection notebook.
- Import the notebook to your workspace.
- Ensure the notebook is in the root of your workspace storage. It should not be in the /helpers folder.
- Start a cluster with Databricks Runtime 13.3 LTS
- Run the notebook.
Once the notebook finishes running, it returns a list of init scripts stored on DBFS in your workspace.
If there are no init scripts stored on DBFS in your workspace, the notebook returns all of the following messages:
No clusters with init scripts on DBFS
No clusters with named init scripts on DBFS
No jobs with init scripts on DBFS
There are no DLT pipelines with init scripts on DBFS
There are no cluster policies with references to init scripts on DBFS
There are no init scripts that reference files on DBFS
Check other workspaces for init scripts stored on DBFS
After running the detection script once in your current workspace a widget is visible at the top of the notebook.
- Enter the PAT for the workspace you want to check into the widget.
- Enter the Workspace URL for the workspace you want to check into the widget.
- Re-run the notebook.
The results displayed apply to the workspace specified in the widget.