IP address limit prevents cluster creation

Learn how to fix a public IP address quota limit Cloud Provider Launch error when starting a Databricks cluster.

Written by laila.haddad

Last published at: May 30th, 2023

Problem

Cluster creation fails with a message about a cloud provider error when you hover over cluster state.

Cloud Provider Launch Failure: A cloud provider error was encountered while setting up the cluster.

When you view the cluster event log to get more details, you see a message about publicIPAddresses limits.

ResourceQuotaExceeded Azure error message: Creating the resource of type 'Microsoft.Network/publicIPAddresses' would exceed the quota of '800' resources of type 'Microsoft.Network/publicIPAddresses' per resource group. The current resource count is '800', please delete some resources of this type before creating a new one.'

You may also see an error about core address limits.

Operation results in exceeding quota limits of Core. Maximum allowed: 350, Current in use: 350, Additional requested: 4.

Cause

Azure subscriptions have a public IP address limit which restricts the number of public IP addresses you can use. This is a hard limit. If you try to start a cluster that would result in your account exceeding the public IP address quota the cluster launch will fail.

Solution

You can either free up resources or request a quota increase for your account.

  • Stop inactive clusters to free up public IP addresses for use.
  • Open an Azure support case with a request to increase the quota limit for your subscription.


For more information, please review the following documentation: