Problem

When running a MERGE transaction, you intermittently encounter the message "No rows returned" in the UI, as shown in the following screenshot, despite the destination table showing records upserted. 

Cause

The writer fails to report metrics properly to the execution context during the MERGE operation. This behavior has been traced to the eager clustering feature in Delta Lake. 

Eager clustering is automatically enabled by default in Databricks Runtime versions below 16.1. The issue can still occur if you explicitly enable eager clustering.

Solution

Databricks recommends upgrading your Databricks Runtime to version 16.1 or above, where this issue has been resolved.

If you are not able to upgrade, disable eager clustering for the affected table by setting the table property delta.liquid.forceDisableEagerClustering to True. This prevents the incorrect reporting of merge operation metrics.

Run the following SQL command in a notebook.

ALTER TABLE <table-name> SET TBLPROPERTIES ('delta.liquid.forceDisableEagerClustering' = 'True');