Task count performance while using serverless compute not showing consistently

Absence of serverless task count performance information at times is expected.

Written by Raghavan Vaidhyaraman

Last published at: October 14th, 2025

Problem

Your task using serverless compute does not consistently display task count performance. In the following screenshot, the task count performance information (in the Tasks column) is visible.

 

In another screenshot of the same notebook view, the task information is not visible.

 

Cause

The task information is absent because the execution completes fast enough that the responses are flushed and the stream ended, rather than waiting on progress updates from callbacks.

 

Alternatively, the driver or thread handling propagation of events may be under heavy load, so listener events are delayed or dropped.

 

Additional context 

On serverless compute, notebooks and jobs use the Apache Spark Connect protocol under the hood. Consequently, every execution operation is a remote procedure call (RPC) to the backing compute clusters. Progress data like task count is collected using a callback mechanism through SparkListeners.

 

The results of executions are embedded in the response stream to the RPC. In addition, the stream may also contain execution progress data such as tasks or stages finished, which the notebook UI visualizes. In the absence of progress data, the column may be hidden. 

 

Solution

This variance is expected behavior. There is no action required or reason to be concerned.