Problem
When you use Conda as your environment manager in MLflow on Databricks Runtime 13.x ML and above, you encounter the following ExecutionException
error. This problem does not occur in Databricks Runtime 12.2 ML LTS.
`ExecutionException: Could not find Conda executable at /databricks/conda/bin/conda. Ensure Conda is installed as per the instructions at <https://conda.io/projects/conda/en/latest/user-guide/install/index.html>. You can also configure MLflow to look for a specific Conda executable by setting the MLFLOW_CONDA_HOME environment variable to the path of the Conda executable.`
Cause
MLflow is still configured to use Conda as your environment manager to manage dependencies, but it is no longer available. The miniconda package has been removed from Databricks Runtime 13.0 ML and above.
Solution
Switch your environment manager for MLflow to virtualenv in your workflow or notebook. Update your env_manager
variable from “conda”
to “virtualenv”
.
Example
mlflow.pyfunc.spark_udf(
model_uri=model_uri,
result_type="double",
env_manager="virtualenv"
)