Add libraries to a job cluster to reduce idle time

How to add libraries to a job cluster and reduce idle time in Databricks

Written by Adam Pavlacka

Last published at: December 4th, 2023



You have an automated job that requires the use of external Maven libraries.

You created a separate cluster with the libraries installed, but it incurs idle time, resulting in unnecessary costs.


To add libraries to a job cluster, follow these steps:

  1. Create a job in Databricks.
  2. Click Add next to dependent libraries.
  3. In the pop-up window, add the required libraries.

To reduce idle time in a job cluster, you have two options:

  • Opt out of auto termination by clearing the Auto Termination checkbox.
  • Specify an inactivity period of 0.

Databricks recommends running jobs on a job cluster, rather than an interactive cluster with auto termination.

Job clusters automatically terminate once the job completes, ensuring efficient resource utilization.

Was this article helpful?