Enable OpenJSSE and TLS 1.3

Add OpenJSSE to allow the use of TLS 1.3 for encrypted data transmission....

How to calculate the number of cores in a cluster

Learn how to calculate the number of cores in a Databricks cluster....

Install a private PyPI repo

How to install libraries from private PyPI repositories....

IP access list update returns INVALID_STATE

Cannot update IP access list. INVALID_STATE error message....

Cannot apply updated cluster policy

When performing an update to an existing cluster policy, the update does not apply unless you remove and re-add the policy....

Cluster Apache Spark configuration not applied

Values set in your cluster's Spark configuration are not applying correctly....

Cluster failed to launch

Learn how to resolve cluster launch failures....

Custom Docker image requires root

Custom Docker containers must be configured to start as the root user when used with Databricks....

Job fails due to cluster manager core instance request limit

Learn how to troubleshoot Databricks errors related to API rate limits....

Admin user cannot restart cluster to run job

Learn how to re-grant privileges to Databricks Admin users....

Cluster fails to start with dummy does not exist error

Cluster is not starting due to a `dummy does not exist` Apache Spark error message....

Cluster slowdown due to Ganglia metrics filling root partition

Resolve cluster slowdowns due to a Ganglia metric data explosion filling the root partition....

Failed to create cluster with invalid tag value

Cluster creation fails if optional tag values do not conform to cloud vendor requirements....

Persist Apache Spark CSV metrics to a DBFS location

Persist Spark CSV metrics to a sink in a DBFS location....

Replay Apache Spark events in a cluster

Use a single node cluster to replay another cluster's event log in the Spark UI....

Set Apache Hadoop core-site.xml properties

Set Apache Hadoop core-site.xml properties in a Databricks cluster....

Set executor log level

Learn how to set the log levels on Databricks executors....

Apache Spark job doesn’t start

Learn how to troubleshoot a Databricks Spark job that won't start....

Auto termination is disabled when starting a job cluster

Auto termination policies are not supported on job clusters....

Unexpected cluster termination

Learn how to troubleshoot a Databricks cluster that stopped unexpectedly....

How to configure single-core executors to run JNI libraries

Learn how to configure single-core executors to run JNI libraries on Databricks....

How to overwrite log4j configurations on Databricks clusters

Learn how to overwrite log4j configurations on Databricks clusters....

Apache Spark executor memory allocation

Understand how Spark executor memory allocation works in a Databricks cluster....

Apache Spark UI shows less than total node memory

Learn what to do when the Spark UI shows less memory than is actually available on the node....

Configure a cluster to use a custom NTP server

Configure your clusters to use a custom NTP server (public or private) instead of using the default server....

Enable GCM cipher suites

Enable AES-GCM encryption (GCM cipher suites) for use with SSL connections to other clusters. Resolve javax.net.ssl.SSLHandshakeException error....

Enable retries in init script

Add a retry function to your init script....

Cannot set a custom PYTHONPATH

Setting a custom PYTHONPATH in an init script or in DCS is not supported....

Run a custom Databricks runtime on your cluster

Configure your cluster to run a custom Databricks runtime image via the UI or API....

Cluster init script fails with mirror sync in progress error

If the mirror you are using is not in sync with the main repository, apt-get update returns a Mirror sync in progress error....

Slow cluster launch and missing nodes

Learn how to resolve a "nodes could not be acquired" error when starting a Databricks ....

IP address limit prevents cluster creation

Learn how to fix a public IP address quota limit Cloud Provider Launch error when starting a Databricks cluster....

CPU core limit prevents cluster creation

Learn how to fix a CPU core quota limit Cloud Provider Launch error when starting a Databricks cluster....

Custom garbage collection prevents cluster launch

Using a custom garbage collection algorithm on Databricks Runtime 10.0 and above prevents the cluster from starting....

SSH to the cluster driver node

How to SSH to the Apache Spark cluster driver node in an Azure virtual network...

Adding a configuration setting overwrites all default spark.executor.extraJavaOptions settings

Learn how to resolve overwritten configuration settings in Databricks....

UnknownHostException on cluster launch

Troubleshoot an UnknownHostException on cluster launch. This is often a DNS configuration issue....

Pin cluster configurations using the API

Pin up to 100 compute cluster configurations using the API....

Unpin cluster configurations using the API

Unpin compute cluster configurations using the API....

R commands fail on custom Docker cluster

R version 4.2.0 changed the way Renviron.site is initialized, so you must set an environment variable when using custom Docker clusters....

Apache Spark UI task logs intermittently return HTTP 500 error

If the Spark property spark.databricks.ui.logViewingEnabled is set to false, you cannot view task logs in the Spark UI....

Legacy global init script migration notebook

Easily migrate your legacy global init scripts to the current global init script framework....