How Delta cache behaves on an autoscaling cluster

Learn how Delta cache behaves on an autoscaling cluster.

Written by Adam Pavlacka

Last published at: May 10th, 2022

This article is about how Delta cache (AWS | Azure | GCP) behaves on an auto-scaling cluster, which removes or adds nodes as needed.

  • When a cluster downscales and terminates nodes:

    A Delta cache behaves in the same way as an RDD cache. Whenever a node goes down, all of the cached data in that particular node is lost. Delta cache data is not moved from the lost node.

  • When a cluster upscales and adds new nodes:

    Whenever a cluster adds a new node, data is not moved between caches. Lost data is re-cached the next time an application accesses the data or tables again.