This article is about how Delta cache (AWS | Azure | GCP) behaves on an auto-scaling cluster, which removes or adds nodes as needed.
- When a cluster downscales and terminates nodes:
A Delta cache behaves in the same way as an RDD cache. Whenever a node goes down, all of the cached data in that particular node is lost. Delta cache data is not moved from the lost node.
- When a cluster upscales and adds new nodes:
Whenever a cluster adds a new node, data is not moved between caches. Lost data is re-cached the next time an application accesses the data or tables again.