When you are trying to set a custom PYTHONPATH environment variable in a cluster-scoped init script, but the values are overridden at driver startup.
Setting a custom PYTHONPATH in an init scripts does not work and is not supported.
Additionally, you cannot set a custom PYTHONPATH when using Databricks Container Services.
You should not try to set a custom PYTHONPATH.
If you need to use custom Python libraries or modules, install the required files to pre-existing directories that are included in the cluster's PYTHONPATH.
This sample code lists all directories in the cluster's PYTHONPATH.
%python import sys print(sys.path)