Remove Log4j 1.x JMSAppender and SocketServer classes from classpath

Databricks recently published a blog on Log4j 2 Vulnerability (CVE-2021-44228) Research and Assessment. Databricks does not directly use a version of Log4j known to be affected by this vulnerability within the Databricks platform in a way we understand may be vulnerable.

Databricks also does not use the affected classes from Log4j 1.x with known vulnerabilities (CVE-2021-4104, CVE-2020-9488, and CVE-2019-17571). However, if your code uses one of these classes (JMSAppender or SocketServer), your use may potentially be impacted by these vulnerabilities.

If your code uses Log4j, you should upgrade to Log4j 2.17 or above.

If you cannot upgrade for technical reasons, you can use a global init script to strip the affected classes from Log4j on cluster start.

Important

Because we do not control the code you run, we cannot guarantee that this solution will prevent Log4j from loading the affected classes in all cases.

Configure the global init script

Note

Running this script is a breaking change for any code that relies on the affected classes.

  1. Go to the Admin Console and click the Global Init Scripts tab.

  2. Click the + Add button.

  3. Enter the name of the script.

  4. Copy the following script into the Script field.

     #!/bin/bash
    
     echo 'Init script to remove certain Log4J 1.x classes, version 1.0 (2021-12-17)'
    
     FILES_TO_DELETE=(
       org/apache/log4j/net/JMSAppender.class
       org/apache/log4j/net/SocketServer.class
     )
    
     find "/databricks" \
         -name '*log4j*.jar' \
         -exec echo -e "\nProcessing {}" \; -exec zip -d {} "${FILES_TO_DELETE[@]}" \;
    
     exit 0
    
    Global init script to remove Log4j 1.x JMSAppender and SocketServer classes from the classpath.
  5. If you have more than one global init script configured for your workspace, you should configure this script to run after your other scripts.

  6. Ensure the Enabled switch is toggled on.

  7. Click Add.

  8. Restart ALL running clusters.

Verify the affected classes are not available

You should run a test on each cluster to ensure the affected classes are not available.

Test 1

You can run an assert check on the affected classes in a notebook.

assert(this.getClass.getClassLoader().getResource("org/apache/log4j/net/JMSAppender.class") == null)
assert(this.getClass.getClassLoader().getResource("org/apache/log4j/net/SocketServer.class") == null)

This sample code runs successfully if you have disabled the affected classes.

This sample code should return an error if you have NOT disabled the affected classes.

Test 2

You can attempt to import the affected classes into a notebook.

import org.apache.log4j.net.JMSAppender
import org.apache.log4j.net.SocketServer

This sample code runs successfully if you have NOT disabled the affected classes.

This sample code should return an error if you have disabled the affected classes.

Caveats

There are some corner cases where you can re-introduce the Log4j 1.x versions of JMSAppender or SocketServer.

Problem

If you install a Maven library with a transitive dependency on Log4j 1.x, all of its classes are re-added to the classpath.

Solution

You can work around this issue by adding Log4j to the Exclusions field when installing Maven libraries.

Install Maven library options showing the Exclusions field.

Problem

If you configure an external Apache Hive metastore, Apache Spark uses Ivy to resolve and download the correct metastore client library, and all of its transitive dependencies, possibly including Log4j 1.x.

To speed up cluster launch, you can cache the downloaded jars on DBFS and use an init script to install from the cache. If you cache jars like this, it is possible that Log4j 1.x may be included.

Solution

You can configure the init script for your external metastore to delete the affected classes.