Databricks Help Center

Main Navigation

  • Help Center
  • Documentation
  • Knowledge Base
  • Community
  • Training
  • Feedback

Libraries

These articles can help you manage libraries in Databricks.

52 Articles in this category

  • Home
  • All articles
  • Libraries

Cannot import module in egg library

The module in the egg library cannot be imported. Easy install, Python....

Last updated: May 11th, 2022 by xin.wang

Cannot import TabularPrediction from AutoGluon

Cannot import TabularPrediction from AutoGluon v0.0.14 due to a namespace collision. Upgrade to AutoGluon v0.0.15....

Last updated: May 11th, 2022 by kavya.parag

Latest PyStan fails to install on Databricks Runtime 6.4

PyStan 3 doesn't install on Databricks Runtime 6.4 ES....

Last updated: May 11th, 2022 by rakesh.parija

Library unavailability causing job failures

Learn how to resolve Databricks job failures caused by unavailable libraries....

Last updated: May 11th, 2022 by Adam Pavlacka

How to correctly update a Maven library in Databricks

Learn how to correctly update a Maven library in Databricks....

Last updated: May 11th, 2022 by Adam Pavlacka

Init script fails to download Maven JAR

Cluster init script fails to download a Maven JAR when trying to install a library....

Last updated: May 11th, 2022 by arvind.ravish

Install package using previous CRAN snapshot

Avoid a package install error by installing from an earlier CRAN snapshot....

Last updated: May 11th, 2022 by darshan.bargal

Install PyGraphViz

Install PyGraphViz with all required dependencies....

Last updated: May 11th, 2022 by pavan.kumarchalamcharla

Install Turbodbc via init script

Install Turbodbc and its dependencies, libboost-all-dev, unixodbc-dev, and python-dev, with an init script....

Last updated: January 6th, 2023 by John.Lourdu

Cannot uninstall library from UI

Learn what to do when you can't uninstall a library using the Databricks user interface....

Last updated: May 11th, 2022 by Adam Pavlacka

Error when installing Cartopy on a cluster

Cartopy installation fails if libgeos and libproj are not installed....

Last updated: May 11th, 2022 by prem.jayaraj

Error when installing pyodbc on a cluster

Learn how to troubleshoot an error when installing pyodbc on a Databricks cluster....

Last updated: May 11th, 2022 by Adam Pavlacka

Libraries fail with dependency exception

Learn why notebook-scoped libraries trigger an Apache Spark dependency exception; return a requirement cannot be satisfied error....

Last updated: May 11th, 2022 by jordan.hicks

Libraries failing due to transient Maven issue

Library resolution failed. Cannot download some libraries due to transient Maven issue....

Last updated: May 11th, 2022 by dayanand.devarapalli

New job fails when adding a library from DBFS or S3

New jobs fail when adding a library from DBFS or S3 storage. Error Uncaught TypeError Cannot read property 'concat' of undefined...

Last updated: May 12th, 2022 by jordan.hicks

Reading .xlsx files with xlrd fails

xlrd no longer supports .xlsx files. Use openpyxl to read .xlsx files....

Last updated: May 12th, 2022 by prakash.jha

Remove Log4j 1.x JMSAppender and SocketServer classes from classpath

Remove Log4j 1.x JMSAppender and SocketServer classes from classpath....

Last updated: May 16th, 2022 by Adam Pavlacka

Python command fails with AssertionError: wrong color format

Resolve a wrong color format AssertionError caused by nbconvert when a Python command fails....

Last updated: May 16th, 2022 by John.Lourdu

PyPMML fails with Could not find py4j jar error

...

Last updated: April 30th, 2025 by arjun.kaimaparambilrajan

TensorFlow fails to import

TensorFlow fails to import if you have an incompatible version of protobuf installed on your cluster....

Last updated: May 16th, 2022 by kavya.parag

Verify the version of Log4j on your cluster

Verify the version of Log4j installed on your cluster and upgrade if required....

Last updated: May 16th, 2022 by Adam Pavlacka

Apache Spark jobs fail with Environment directory not found error

Spark jobs appear to time out after you install a library because security rules are preventing workers from resolving the Python executable path....

Last updated: July 1st, 2022 by Adam Pavlacka

Use Databricks Repos with Docker container services

Configure your cluster with a custom init script to use Databricks Repos with Docker container services....

Last updated: May 10th, 2023 by darshan.bargal

Copy installed libraries from one cluster to another

Copy libraries from a source cluster to a target cluster with a custom Python script....

Last updated: January 6th, 2023 by manoj.hegde

Failed to install Elasticsearch via Maven

If library dependencies are already installed, it can result in a library installation failure....

Last updated: March 17th, 2023 by ankitha.vijayanandana

Cluster fails to start with InvalidGroup.NotFound error

If the network security group policy is not correctly configured your clusters will fail to start....

Last updated: December 21st, 2023 by Adam Pavlacka

Add libraries to a job cluster to reduce idle time

How to add libraries to a job cluster and reduce idle time in Databricks...

Last updated: December 4th, 2023 by Adam Pavlacka

PyArrow hotfix breaking change

PyArrow versions 0.14 - 14.0.0 contain a security vulnerability....

Last updated: December 6th, 2023 by Adam Pavlacka

OpenSSL SSL_connect: SSL_ERROR_SYSCALL error

Use a cluster-scoped init script to install necessary SSL certificates to resolve a SSL_ERROR_SYSCALL error....

Last updated: February 29th, 2024 by pavan.kumarchalamcharla

Notebook cells fail to run with "Failure Starting repl." and Pandas "check_dependencies" errors

Ensure you do not have a dependency mismatch with the NumPy and/or Pandas versions installed on your cluster....

Last updated: June 21st, 2024 by jairo.prado

Virtualenv creation failure due to setuptools >= 71.0.0

Pin setuptools version 70.3.0....

Last updated: September 12th, 2024 by Cedric Law

Libraries failing with owner or network errors on Databricks Runtime 13.3 LTS - current (15.3)

Manually adjust your custom index URL. ...

Last updated: September 12th, 2024 by david.vega

Maven Libraries Start Failing with Timed-Out Errors When Updating to Databricks Runtime 11.3 LTS - 15.3 (current)

Whitelist Maven Central and the new Maven repo....

Last updated: September 12th, 2024 by david.vega

Paths behave differently on Git folders and workspace folders

Git folders can reference the project root, while workspace folders reference the current working directory....

Last updated: October 18th, 2024 by caio.cominato

GDAL library installation

Troubleshooting GDAL init script issues...

Last updated: November 18th, 2024 by julian.campabadal

Installing lme4 fails with a Matrix version error

Upgrade the version of Matrix to 1.6.2 or above before installing lme4....

Last updated: December 2nd, 2024 by alberto.umana

User not found error while trying to install a library on a shared cluster

Change cluster ownership and reconfigure all libraries under the new owner....

Last updated: December 3rd, 2024 by guruprasad.bn

Init script to set up Dask library fails and cluster won’t start

Modify the initialization script to include a validation check for the required environment variables first....

Last updated: December 20th, 2024 by julian.campabadal

Office365 library installation causes numpy.dtype size change error while executing notebook commands

Pin the Moviepy library version that uses the NumPy version compatible with your Databricks Runtime version....

Last updated: December 24th, 2024 by alberto.umana

Fixture not found error when using pytest on a cluster

Downgrade pytest to version 8.3.2 or upgrade Databricks Runtime to 16.1 or above....

Last updated: February 19th, 2025 by kaushal.vachhani

Error when attempting to use 'stream_read_table' function from Sparklyr in 14.3 LTS ML

Install Sparklyr version 1.8.6 in the cluster's libraries, restart the cluster, and rerun the function to ensure the error is fixed....

Last updated: January 7th, 2025 by Amruth Ashoka

Library installation attempted on the driver node of the cluster failed

Uninstall and reinstall the library. ...

Last updated: January 17th, 2025 by shashank.chaudhary

Py4JJavaError when trying to install libraries on SSL-encrypted cluster

Set the Apache Spark configuration to disable SSL....

Last updated: January 28th, 2025 by vidya.sagamreddy

Unable to install R package 'Survminer' on cluster

Use an init script to install a compatible version of R and the necessary dependencies....

Last updated: January 30th, 2025 by monica.cao

DLTImportException error when importing the DLT module

Use a cluster-scoped init script targeting the job or cell commands in a notebook. ...

Last updated: January 31st, 2025 by Ernesto Calderón

ClassNotFoundException error when executing a job or notebook with a custom Kryo serializer

Use an init script or use the spark.jars property in your Apache spark configuration....

Last updated: May 7th, 2025 by pavan.kumarchalamcharla

Error during Maven library installation: ERROR_MAVEN_LIBRARY_RESOLUTION

Install libraries separately, host a private Maven mirror, or if necessary use another Maven mirror....

Last updated: February 12th, 2025 by alberto.umana

Error when attempting to install torch for R package

Install the required dependencies before installing torch for R....

Last updated: March 19th, 2025 by jairo.prado

You have external location access, but cannot install a JAR on a shared cluster from an S3 path outside of volumes

Use an Instance profile with proper access OR store the JAR in a Unity Catalog volume....

Last updated: March 27th, 2025 by kingshuk.das

Getting ValueError when trying to import PMML files using PyPMML

Remove -XX:+PrintFlagsFinal flag from Java options....

Last updated: April 25th, 2025 by Amruth Ashoka

Getting timeout error during Maven library installation on Databricks cluster

Configure your cluster to use private repos as the default repository, and disable the default Maven Central resolver and Apache Spark packages resolver....

Last updated: April 28th, 2025 by sravya.tanguturi

Notebook or workflow fails with “Error : Py4JError: Could not find py4j jar at” error after trying to install PyPMML on a cluster

Install your Py4J library into the expected location....

Last updated: May 5th, 2025 by Amruth Ashoka

Contact Us

If you still have questions or prefer to get help directly from an agent, please submit a request. We’ll get back to you as soon as possible.

Please enter the details of your request. A member of our support staff will respond as soon as possible.


© Databricks 2022-2025. All rights reserved. Apache, Apache Spark, Spark, and the Spark logo are trademarks of the Apache Software Foundation.

Send us feedback | Privacy Notice (Updated) | Terms of Use | Your Privacy Choices | Your California Privacy Rights Privacy Rights icon


Knowledge Base Software powered by Helpjuice

Definition by Author

0
0