On Dec 1, 2023, Databricks will disable legacy global init scripts for all workspaces. This type of init script was deprecated in 2020 and will not be usable after Dec 1, 2023. The legacy global init script was replaced in 2020 by the more reliable current global init script framework, which continues to be supported.

Databricks recommends that you migrate your legacy global init scripts to the current global init script framework as soon as possible.

You can follow the documentation to manually Migrate from legacy to new global init scripts (AWS | Azure). Alternatively, Databricks Engineering has created a notebook to help automate the migration process.

Instructions

Prerequisites

Before running the migration notebook, you need to have the scope name and secret name for your Personal Access Token.

For more information, please review the Create a Databricks-backed secret scope (AWS | Azure) and the Create a secret in a Databricks-backed scope (AWS | Azure) documentation.

Do a dry run

Executing a dry run allows you to test the migration notebook in your workspace without making any changes.

1. Download the Migrate legacy global init scripts notebook.
2. Import the notebook to your workspace.
3. Start a cluster.
4. Run the notebook.
5. A UI screen appears after you run the notebook. Enter the Scope Name and Secret Name into the appropriate fields.
   
6. After updating the settings, run the notebook a second time.

If there are no errors, you are ready to migrate your legacy global init scripts.

Migrate your legacy global init scripts

You should always do a dry run before making changes to your system.

1. Run the Migrate legacy global init scripts notebook.
2. A UI screen appears after you run the notebook. Enter the Scope Name and Secret Name into the appropriate fields.
3. In the Dry Run drop down menu, select False. This allows the notebook to make changes to your workspace.
4. In the Enable New Scripts drop down menu, select True. This enables the current global init script framework on your workspace. Select False if you want to migrate the scripts, but not enable them.
5. In the Remove Old Scripts drop down menu, select True. This removes the legacy global init scripts.
6. In the Revert Changes drop down menu, select False. This should only be set to True if you need to undo changes for a specific reason.
   
7. After updating the settings, run the notebook a second time.

Once the notebook finishes running, all of your legacy global init scripts are migrated.

Validate the migrated scripts

1. Open the Admin Console.
2. Click Global init scripts.
3. Verify that all of your legacy global init scripts are present.
4. Toggle the Enabled switch to turn individual scripts on or off.
5. Click Edit ordering to change the order of the scripts.
6. Restart your clusters and verify that they run as expected.