Problem

The Corporate Information Security (CIS) Vulnerability Management team identifies vulnerabilities in AWS instances that are traced to EC2 instances created by Databricks (worker AMI).

Cause

The Databricks security team addresses all critical vulnerabilities and updates the core and worker AMIs on a regular basis.

However, if there are long-running clusters that have not been restarted, the newest AMIs don’t get picked up by the cluster EC2 instances. Therefore, a scan might find vulnerabilities.

Solution

Restart long-running clusters or start a brand new cluster when you do vulnerability scans against specific Databricks worker EC2 instances.