Job fails due to job rate limit

Learn how to resolve Databricks job failures due to job rate limits.

Written by Adam Pavlacka

Last published at: April 17th, 2023

Problem

A Databricks notebook or Jobs API request returns the following error:

Error : {"error_code":"INVALID_STATE","message":"There were already 1000 jobs created in past 3600 seconds, exceeding rate limit: 1000 job creations per 3600 seconds."}

You are not able to run jobs getting a QUOTA_EXCEEDED error message:

'error_code':'QUOTA_EXCEEDED','message':'The quota for the number of jobs has been reached. The current quota is 1000. This quota is only applied to jobs created through the UI or through the /jobs/create endpoint, which are displayed in the Jobs UI

Cause

This error occurs because the number of jobs per hour exceeds the limit of 1000 established by Databricks to prevent API abuses and ensure quality of service.

Solution

If you cannot ensure that the number of jobs created in your workspace is less than 1000 per hour, contact Databricks Support to request a higher limit. A job rate limit increase requires at least 20 minutes of downtime.

  • Databricks can increase the job limit maximumJobCreationRate up to 2000.
  • Currently running jobs will be affected while the limit is being increased.