dnsmasq is a tool for installing and configuring DNS routing rules for cluster nodes. You can use it to set up routing between your Databricks environment and your on-premise network.

Use the following cluster-scoped init script to configure dnsmasq for a cluster node.

1. Use netcat (nc) to test connectivity from the notebook environment to your on-premise network.

   nc -vz <on-premise-ip> 53

2. Create the base directory you want to store the init script in if it does not already exist.

   dbutils.fs.mkdirs("dbfs:/databricks/<init-script-folder>/")

3. Create the script.

   AWS Scala example

   dbutils.fs.put("/databricks/<init-script-folder>/dns-masq.sh","""
   #!/bin/bash
   ########################################
   Configure on-prem dns access.
   ########################################
   
   sudo apt-get update -y
   sudo apt-get install dnsmasq -y --force-yes
   
   ## Add dns entries for internal your-company.net name servers
   echo server=/databricks.net/<dns-server-ip> | sudo tee --append /etc/dnsmasq.conf
   
   ## Find the default DNS settings for the EC2 instance and use them as the default DNS route
   
   ec2_dns=cat /etc/resolv.conf | grep "nameserver"; | cut -d' ' -f 2
   echo "Old dns in resolv.conf $ec2_dns"
   
   echo "server=$ec2_dns" | sudo tee --append /etc/dnsmasq.conf
   
   ## configure resolv.conf to point to dnsmasq service instead of static resolv.conf file
   mv /etc/resolv.conf /etc/resolv.conf.orig
   echo nameserver 127.0.0.1 | sudo tee --append /etc/resolv.conf
   sudo systemctl disable --now systemd-resolved
   sudo systemctl enable --now dnsmasq
   """, true)

   Delete

   Azure Scala example

   dbutils.fs.put("/databricks/<init-script-folder>/dns-masq.sh","""
   #!/bin/bash
   sudo apt-get update -y
   sudo apt-get install dnsmasq -y --force-yes
   
   ## Add dns entries for internal nameservers
   echo server=/databricks.net/<dns-server-ip> | sudo tee --append /etc/dnsmasq.conf
      
   ## Find the default DNS settings for the instance and use them as the default DNS route
   azvm_dns=cat /etc/resolv.conf | grep "nameserver"; | cut -d' ' -f 2
   echo "Old dns in resolv.conf $azvm_dns"
   echo "server=$azvm_dns" | sudo tee --append /etc/dnsmasq.conf
       
   ## configure resolv.conf to point to dnsmasq service instead of static resolv.conf file
   mv /etc/resolv.conf /etc/resolv.conf.orig
   echo nameserver 127.0.0.1 | sudo tee --append /etc/resolv.conf
   sudo systemctl disable --now systemd-resolved
   sudo systemctl enable --now dnsmasq
   """, true)

   Delete
4. Check that the script exists.

   display(dbutils.fs.ls("dbfs:/databricks/<init-script-folder>/dns-masq.sh"))

5. Configure the init script that you just created as a cluster-scoped init script. You will need the full path to the location of the script (dbfs:/databricks/<init-script-folder>/dns-masq.sh).
6. Launch a zero-node cluster to confirm that you can create clusters.