Problem 

When trying to create a compute cluster, you encounter a "Security Daemon Registration Exception" error. 

Error: Security Daemon Registration Exception:
Failed to set up the Spark container due to an error when registering the container to security daemon.

Cause 

The instance cannot access the Security Token Service (STS) endpoint. Several factors can contribute to an inability to access:

1. Missing VPC endpoint for STS in the Databricks-managed VPC.
2. Misconfigured regional STS endpoints, not linked to the appropriate Databricks subnets and security groups.
3. Network configuration changes that inadvertently block access to the STS endpoint.

Solution

1. If it is not yet set up, create a VPC endpoint for STS in the Databricks-managed VPC. Please review the instructions in the Configure a customer-managed VPC documentation.
2. Ensure that regional STS endpoints are associated with the correct subnets and security groups within Databricks. Work with your network team to confirm there have been no recent changes to network configurations that may have caused issues.
3. Review the network configuration to ensure no rules or settings are blocking access to the STS endpoint. Consult your network team to make any necessary adjustments.

Additionally, to validate your network, follow the troubleshooting instructions in the Create a workspace using the Account API documentation.