Cannot import module in egg library

Problem

You try to install an egg library to your cluster and it fails with a message that the a module in the library cannot be imported.

Even a simple import fails.

import sys
egg_path='/dbfs/<path-to-egg-file>/<egg-file>.egg'
sys.path.append(egg_path)
import shap_master

Cause

This error message occurs due to the way the library is packed.

Solution

If the standard library import options do not work, you should use easy_install to import the library.

dbutils.fs.put("/<path>/<library-name>.sh","""
#!/bin/bash
easy_install-3.7 /dbfs/<path-to-egg-file>/<egg-file>.egg""")
""")

Important

The version of easy_install must match the version of Python on the cluster. You can determine the version of Python on your cluster by reviewing the release notes.