Problem

When a user who has permission to start a cluster, such as a Databricks Admin user, submits a job that is owned by a different user, the job fails with the following message:

Message: Run executed on existing cluster ID <cluster id> failed because of insufficient permissions. The error received from the cluster manager was: 'You are not authorized to restart this cluster. Please contact your administrator or the cluster creator.'

Cause

This error can occur when the job owner’s privilege to start the cluster is revoked. In this scenario, the job will fail even if it is submitted by an Admin user.

Solution

Re-grant the privilege to start the cluster (known as Can Manage) to the job owner. Change the job owner to a user or group that has the cluster start privilege. You can change it by navigating to your job page in Jobs, then to Advanced > Permissions > Edit.