Problem
When you try to install Maven libraries in your Databricks cluster, you experience long installation times and then timeout errors. The error message appears as either of the following.
Library installation timed out after 1800 seconds.
Or
failed: Network is unreachable (connect failed)
Cause
Your workspace has restrictions on public network access, which is blocking the package installer from reaching the public Maven repository.
Solution
Configure your cluster to use a private repo as the default repository and disable the default Maven Central resolver. Changing these settings forces the package installer to use the on-premise artifactory as the default repository, bypassing the need to reach the public Maven repo.
In your cluster settings, navigate to the Advanced options > Spark tab. In the Spark config field, add the following settings.
1. Configure the default repository to your private repo.
spark.databricks.driver.preferredMavenCentralMirrorUrl <your-repo>
2. Disable the default Maven Central resolver.
spark.databricks.driver.disableDefaultMavenCentralResolver true
3. Disable the Apache Spark packages resolver.
spark.databricks.driver.disableSparkPackagesResolver true