Problem

When you try to run SQL queries in Classic or Pro SQL warehouses, they fail with the following error message. 

MetadataFetchFailedException:
org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 2549 partition 49

Cause

While your query was executing, the spot instance terminated and the worker nodes were lost.

When a worker node is terminated, any shuffle files stored on that node are also lost causing the stage to fail with a shuffle fetch failure.

Solution

Update your spot instance policy to Reliability Optimized. 

• Navigate to the SQL Warehouse where you executed the query.
• Click Edit.
• Under Advanced options, change the Spot Instance Policy from Cost Optimized to Reliability Optimized.
   

This setting ensures that all nodes are launched as on-demand instances, significantly reducing the risk of unexpected termination during query execution.