Databricks SQL Warehouse query finish with cache is not populating ‘open in Spark UI’ option

When a query uses 100% of the cache, there are no corresponding Spark jobs or stages to display in Spark UI.

Written by Rajeev kannan Thangaiah

Last published at: January 31st, 2025

Problem

When working with Databricks SQL, typically when queries finish you can click the kebab menu and choose Open in Spark UI from the dropdown menu. However, for queries that finish with cache, you notice the Open in Spark UI option is missing from the menu.

 

Cause

The Spark UI displays information about Spark jobs and stages. When a query uses 100% of the cache, there are no corresponding Apache Spark jobs or stages launched, so there is no relevant information to display. The menu option Open in Spark UI is not applicable.

 

Solution

To access the Spark UI for the corresponding warehouse instead: 

  1. Select any query that has run and triggered Spark jobs or stages. 
  2. Use that query's URL page to access the Spark UI.