Monitor running jobs with a Job Run dashboard

The Job Run dashboard is a notebook that displays information about all of the jobs currently running in your workspace.

To configure the dashboard, you must have permission to attach a notebook to an all-purpose cluster in the workspace you want to monitor. If an all-purpose cluster does not exist, you must have permission to create one.

Once the dashboard is configured, you can manage job permissions and assign Can View permissions to users in your organization. These users can view the dashboard, but cannot modify it.

Job Run dashboard notebook

Open notebook in new tab

Attach the dashboard

Because the Job Run dashboard is a notebook, no special steps are required to attach the notebook to a cluster.

Attach it to an all-purpose cluster.

Run the dashboard as a scheduled job

After attaching the notebook to a cluster in your workspace, configure it to run as a scheduled job that runs every minute.

  1. Open the notebook.
  2. Click Schedule in the notebook toolbar.
  3. Click New in the Schedule job pane.
  4. Select Every and minute in the Create Schedule dialog box.
  5. Click OK.
  6. Click Job Run dashboard in the Schedule job pane.
  7. Click Edit next to the Cluster option on the job details page.
  8. Select an existing all-purpose cluster.
  9. Click Confirm.

Display dashboard

  1. Go to the job details page for the scheduled job.
  2. Check to make sure at least one successful run has occurred.
  3. Click Latest successful run (refreshes automatically).
  4. Select the Job Run Dashboard view.

The dashboard is now in presentation mode. It updates automatically after each scheduled run completes.

You can share the dashboard URL with any user who has view permissions.

Results listed

The Job Run dashboard results are split into two sections:

  • Job Runs - Displays all of the scheduled jobs that are currently running.
  • Run Submits - Displays all of the running jobs that were invoked via an API call.

The dashboard displays the following components for each job:

  • Job ID - This is the unique ID number for the job. You can use this to view all of the job data by entering it into a job URL.
  • Run Page - This is the ID number of the specific run for a given job. It is formatted as a clickable hyperlink, so you can navigate directly to the run page from the Job Run dashboard. You can access previous run pages by navigating to the job URL and then clicking the specific run page from the list of completed runs.
  • Run Name - This is the name of the notebook associated with the job.
  • Start Time - This is the time the job run began. Time is displayed in DD-MM-YYYY HH:MM:SS format, using a 24 hour clock. Time is in UTC.
  • Created By - This is the email address of the user who owns the job.