Problem

While creating an Azure Key Vault (AKV)-backed secret scope, you don’t set permissions for other users to access it. When you try to go back and change the setting to All Users via the UI, you encounter a “permission denied” error. 

Cause

When creating AKV-backed secret scopes in Databricks, the 'Manage Principal' default is set to 'Creator', meaning only the creator and workspace admins can list and read the secrets from the scope. 

Non-admin users, even if they have the required permissions on the Azure side, cannot access the secrets due to the independent nature of Databricks' Access Control List (ACL) and AKV's access policies. 

Solution

Use the Databricks CLI to edit the Manage Principal value for the existing secret scope. 

The following command can be used to grant read permissions to all users in the Databricks workspace. The principal can be a user, a group, or a service principal. The permission parameters are all capitalized (READ, WRITE, MANAGE). 

databricks secrets put-acl --scope <scope-name> --principal users --permission READ

For more information, refer to the Secrets documentation. 

Alternatively, create a new secret scope with the desired Manage Principal setting. 

Whether you edit an existing secret scope or create a new one, consider creating a group with the desired users and granting the necessary permissions to the group. This simplifies permission management and ensures that all relevant users have access to the secret scope.

For more detail on managing secrets and ACLs via the Databricks API, refer to the Secret API documentation.