Problem
When you try to map a Databricks secret scope to Azure Key Vault using a Databricks-managed service principal in Terraform, you receive the following error.
Error Msg:
│ Error: cannot create secret scope: Scope with Azure KeyVault must have userAADToken defined!
Cause
The Databricks-managed service principal cannot access the Azure Key Vault.
Solution
Use Terraform to create the secret scope in Azure Key Vault directly instead.
- Create an Azure-based service principal using Terraform. Follow the steps provided in the Terraform databricks_service_principal Resource documentation. The following code is an example.
resource "databricks_service_principal" "sp" {
application_id = "<your-application-id>"
display_name = "<your-service-principal-display-name>"
allow_cluster_create = true
}
- Once you have created the service principal, use it to authenticate with Databricks. For detailed steps on how to authenticate access to Databricks resources, refer to the Databricks Authorizing access to Databricks resources documentation.
- Use the Azure-managed service principal for authentication you created in step 1 to create an Azure Key Vault-backed secret scope in Databricks. For instructions, refer to the Create an Azure Key Vault-backed secret scope section of the Secret management documentation.
Supporting details
- Azure Key Vault-backed secret scopes are read-only interfaces to the Azure Key Vault. You must use an Azure-managed service principal to create and manage these secret scopes.
- Personal Access Tokens (PATs) cannot be used to create Azure Key Vault-backed secret scopes. You need to use one of the Azure-specific authentication methods.
- If you encounter errors while creating the secret scope using Terraform, ensure that the service principal has the necessary permissions on the Azure Key Vault. For more information, review the requirements in the Secret management documentation.