Troubleshooting unresponsive Python notebooks or canceled commands

Learn how to troubleshoot unresponsive Python notebooks and cancelled commands in Databricks notebooks.

Written by Adam Pavlacka

Last published at: May 17th, 2022

This article provides an overview of troubleshooting steps you can take if a notebook is unresponsive or cancels commands.

Check metastore connectivity

Problem

Simple commands in newly-attached notebooks fail, but succeed in notebooks that were attached to the same cluster earlier.

Troubleshooting steps

  1. Check metastore connectivity. The inability to connect to the Hive metastore can cause REPL initialization to hang, making the cluster appear unresponsive.
  2. Are you are using the Databricks metastore or your own external metastore? If you are using an external metastore, have you changed anything recently? Did you upgrade your metastore version? Rotate passwords or configurations? Change security group rules?

See Metastore for more troubleshooting tips and solutions.

Check for conflicting libraries

Problem

Python library conflicts can result in cancelled commands. The Databricks support organization sees conflicts most often with versions of ipython, numpy, scipy, and pandas.

Troubleshooting steps

Review the Cluster cancels Python command execution due to library conflict KB article for more information.

For more notebook troubleshooting information, see Notebooks.