• Home
  • All articles
  • Unity Catalog
  • Error listDatabases() method is not allowlisted when running CatalogImpl.listDatabases() on a no isolation shared cluster

Error listDatabases() method is not allowlisted when running CatalogImpl.listDatabases() on a no isolation shared cluster

Update to Databricks Runtime 14.1 or above.

Written by jose.salgado

Last published at: March 20th, 2025

Problem

When you attempt to run the spark.catalog.listDatabases() command in a no isolation shared (formerly high concurrency) cluster, you receive a py4j.security.Py4JSecurityException error, indicating that the listDatabases method in the CatalogImpl class is not allowlisted.

 

Cause

No isolation shared (formerly high concurrency) clusters have stricter security settings. The listDatabases method is not allowlisted by default in these clusters, leading to the observed error. 

 

Solution

Use Databricks Runtime 14.1 or above to use spark.catalog functions in shared clusters for Python and Scala.

 

If you need to work with a Databricks Runtime below 14.1, switch to a dedicated (formerly single user) cluster. 

 

Preventative measures

  • Regularly review and update your cluster configurations to ensure they meet your needs and comply with Databricks security recommendations.
  • Stay up-to-date with the latest Databricks runtime versions and releases to take advantage of new features, bug fixes, and security improvements. For details, review the Databricks Runtime release notes versions and compatibility (AWS | Azure | GCP) documentation.