By default Databricks clusters use public NTP servers. This is sufficient for most use cases, however you can configure a cluster to use a custom NTP server. This does not have to be a public NTP server. It can be a private NTP server under your control. A common use case is to minimize the amount of Internet traffic from your cluster.
Update the NTP configuration on a cluster
- Create a ntp.conffile with the following information:
# NTP configuration server <ntp-server-hostname> iburstwhere <ntp-server-hostname> is a NTP server hostname or a NTP server IP address.
If you have multiple NTP servers to list, add them all to the file. Each server should be listed on its own line.
- Upload the ntp.conf file to /dbfs/databricks/init_scripts/ on your cluster.
- Create the script ntp.shon your cluster:
%python dbutils.fs.put("/databricks/init_scripts/ntp.sh",""" #!/bin/bash echo "<ntp-server-ip> <ntp-server-hostname>" >> /etc/hosts cp /dbfs/databricks/init_scripts/ntp.conf /etc/ sudo service ntp restart""",True)
- Confirm that the script exists:
- Click Clusters, click your cluster name, click Edit, click Advanced Options, click Init Scripts.
- Select DBFS under Destination.
- Enter the full path to ntp.sh and click Add.
- Click Confirm and Restart. A confirmation dialog box appears. Click Confirm and wait for the cluster to restart.
Verify the cluster is using the updated NTP configuration
- Run the following code in a notebook:
%sh ntpq -p
- The output displays the NTP servers that are in use.