Configure a cluster to use a custom NTP server
By default Databricks clusters use public NTP servers. This is sufficient for most use cases, however you can configure a cluster to use a custom NTP server. This does not have to be a public NTP server. It can be a private NTP server under your control. A common use case is to minimize the amount of Internet traffic from your cluster.
Update the NTP configuration on a cluster
Create a
ntp.conf
file with the following information:# NTP configuration server <ntp-server-hostname> iburst
where
<ntp-server-hostname>
is a NTP server hostname or a NTP server IP address.If you have multiple NTP servers to list, add them all to the file. Each server should be listed on its own line.
Upload the
ntp.conf
file to/dbfs/databricks/init_scripts/
on your cluster.Create the script
ntp.sh
on your cluster:dbutils.fs.put("/databricks/init_scripts/ntp.sh",""" #!/bin/bash echo "<ntp-server-ip> <ntp-server-hostname>" >> /etc/hosts cp /dbfs/databricks/init_scripts/ntp.conf /etc/ sudo service ntp restart""",True)
Confirm that the script exists:
display(dbutils.fs.ls("dbfs:/databricks/init_scripts/ntp.sh"))
Click Clusters, click your cluster name, click Edit, click Advanced Options, click Init Scripts.
Select DBFS under Destination.
Enter the full path to
ntp.sh
and click Add.Click Confirm and Restart. A confirmation dialog box appears. Click Confirm and wait for the cluster to restart.