Cannot import module in egg library
Problem
You try to install an egg library to your cluster and it fails with a message that the a module in the library cannot be imported.
Even a simple import fails.
import sys
egg_path='/dbfs/<path-to-egg-file>/<egg-file>.egg'
sys.path.append(egg_path)
import shap_master
Solution
If the standard library import options do not work, you should use easy_install
to import the library.
dbutils.fs.put("/<path>/<library-name>.sh","""
#!/bin/bash
easy_install-3.7 /dbfs/<path-to-egg-file>/<egg-file>.egg""")
""")
Important
The version of easy_install
must match the version of Python on the cluster. You can determine the version of Python on your cluster by reviewing the release notes.