Problem
While attempting to run job clusters using a service principal, you receive the error:
You cannot set the job's identity to <SP ID> because you do not have the required permissions. Please contact your workspace administrator or the user who manages the service principal.
Additionally, you may see PERMISSION_DENIED: Please contact your administrator.
Cause
The job clusters are configured to run as a service principal, but the necessary permissions are not correctly set.
Solution
Ensure that the service principal has the 'Service Principal User' role.
- Explicitly assign yourself the service principal user role, even after creating the service principal. (Manager does not automatically inherit User permissions.)
- Indicate 'can use' permission on the cluster policy.
For additional information, please refer to the Roles for managing service principals (AWS | Azure | GCP) documentation.