Creating an Azure Key Vault-backed secret scope with the Databricks CLI fails with a userAADToken error

Ensure the authentication is properly configured in the .databrickscfg file.

Written by rushali.kumari

Last published at: May 27th, 2025

Problem

You are attempting to create an Azure Key Vault-backed secret scope using the Databricks CLI, but the operation fails with an AAD token definition error message.

"Scope with Azure KeyVault must have userAADToken defined!"

 

Cause

This error occurs when the authentication is not properly configured in the .databrickscfg file for the service principal. You did not define the necessary userAADToken.

 

Solution

Follow the steps below to create an Azure Key Vault-backed secret scope using the Databricks CLI.

Note

Creating an Azure Key Vault-backed secret scope requires the Contributor or Owner role on the Azure key vault instance, even if the Azure Databricks service has previously been granted access to the key vault. Ensure the service principal has the necessary roles assigned. For more information, review the Azure Key Vault-backed secret scope requirements documentation.

 

 

1. Create or update the .databrickscfg file to include a profile for the service principal. Use the following format:


[DEFAULT] 
	host = <databricks-instance-url> 
	azure_workspace_resource_id = <databricks-workspace-resource-id>
	tenant_id = <azure-service-principal-tenant-id> 
	client_id = <service-principal-application-id> 
	client_secret = <azure-service-principal-client-secret> 

 

Where: 

  • <databricks-instance-url> – The URL of your Databricks workspace. 
  • <databricks-workspace-resource-id> – The Azure resource ID of your Databricks workspace, available in the Azure portal under Resource JSON.
  • <azure-service-principal-tenant-id> – The Directory (tenant) ID associated with the Azure Active Directory (Microsoft Entra ID) where the service principal is registered.
  • <service-principal-application-id> – The Application (client) ID of your Azure service principal.
  • <azure-service-principal-client-secret> – The client secret value generated when you created client credentials.

           
For more information, review the Microsoft Entra ID service principal authentication documentation.

 

2. Authenticate in the Databricks CLI using the service principal.     

databricks auth env --profile DEFAULT

 

3. Create a JSON configuration file.

{
  "scope": "<scope-name>",
  "initial_manage_principal": "users",
  "scope_backend_type": "AZURE_KEYVAULT",
  "backend_azure_keyvault": {
  "resource_id": "<resource-id>",
  "dns_name": "<dns-name>"
        }
}

 

Where:

  • <scope-name> – A name for the secret scope you are creating.
  • <resource-id> – Available in the Overview section of your Key Vault >JSON View. Copy the Resource ID and paste in the above code.
  • <dns-name> – Also available in the Overview section of your Key Vault > JSON View > Properties. Copy the vaultUri and paste in the above code. 

 

4. Create the secret scope with the Databricks CLI.

databricks secrets create-scope --json @path_to_json_file.json

 

5. List all secret scopes to verify the new secret scope was successfully created.

databricks secrets list-scopes