Jobs are not progressing in the workspace

Learn how to troubleshoot a Databricks workspace in a virtual network using Azure Firewall. "WARN TaskSchedulerImpl Initial job has not accepted any resources" error message.

Written by Adam Pavlacka

Last published at: December 7th, 2022

Problem

Jobs fail to run on any cluster in the workspace.

Cause

This can happen if you have changed the VNet of an existing workspace. Changing the VNet of an existing Azure Databricks workspace is not supported.

Review Deploy Azure Databricks in your Azure virtual network (VNet injection) for more details.

Solution

  1. Open the cluster driver logs in the Azure Databricks UI.
  2. Search for the following WARN messages:
    19/11/19 16:50:29 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
19/11/19 16:50:44 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
19/11/19 16:50:59 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
    If this error is present, it is likely that the VNet of the Azure Databricks workspace was changed.
  3. Revert the change to restore the original VNet configuration that was used when the Azure Databricks workspace was created.
  4. Restart the running cluster.
  5. Resubmit your jobs.
  6. Verify the jobs are getting resources.