Problem

You may encounter difficulties when trying to trigger a job with a specific user ID using the Run as option. This issue arises when multiple users need to run jobs with their own input values and then later filter the job runs based on the user who initiated them.

Cause

The Jobs API does not support changing the Run as parameter for each job run. This prevents users from specifying their own user ID when triggering a job, making it challenging to filter job runs by the initiating user.

By default, the job creator is tagged as the Run as user. We can edit the Run as a user to run a job as a service principal (AWS | Azure | GCP), but it is a one-time activity. Each run cannot normally have an individual identity.

Solution

The identity assigned should have the following permissions:

• Cluster creation (if using the job cluster)
• Permission to execute and run the underlying resources (interactive cluster, notebook, Python file, other job resources)

There are two ways to trigger a job with a specific Run as user.

Use a Run Job task in the UI

1. Click New.
2. Click Job.
3. In the Tasks tab, set Run Job (AWS | Azure | GCP) as the Type.
4. In the Job drop-down menu, select the job you want to execute. This task type uses the same job configuration as the original job but triggers it with the user ID specified in the Job details field.

Use the API

1. Use the create command (AWS | Azure | GCP) in the Jobs API payload to trigger the job with your desired Run as user.
2. Use the sample code and update the following values:
   • <job-name>: The desired job name.
   • <task-key-name>: The desired task name.
   • <job-id-to-be-triggered>: The job_id to trigger with the new job.
   • <user-name@domain.com>: User name to be assigned.

The Run as job uses the same configuration as the parent job.

Example code

{
   "name": "<job-name>",
   "email_notifications": {
       "no_alert_for_skipped_runs": false
   },
   "webhook_notifications": {},
   "timeout_seconds": 0,
   "max_concurrent_runs": 1,
   "tasks": [
       {
           "task_key": "<task-key-name>",
           "run_if": "ALL_SUCCESS",
           "run_job_task": {
               "job_id": "<job-id-to-be-triggered>"
           },
           "timeout_seconds": 0,
           "email_notifications": {}
       }
   ],
   "queue": {
       "enabled": true
   },
   "run_as": {
       "user_name": "<user-name@domain.com>"
   }
}

The payload triggers the root job with job_id as <job-id-to-be-triggered>, using the specified user name. This allows you to filter job runs in the Run as section of the Job Run UI.