By default Databricks clusters use public NTP servers. This is sufficient for most use cases, however you can configure a cluster to use a custom NTP server. This does not have to be a public NTP server. It can be a private NTP server under your control. A common use case is to minimize the amount of Internet traffic from your cluster.
ntp.conffile with the following information:
# NTP configuration server <ntp-server-hostname> iburst
<ntp-server-hostname>is a NTP server hostname or a NTP server IP address.
If you have multiple NTP servers to list, add them all to the file. Each server should be listed on its own line.
/dbfs/databricks/init_scripts/on your cluster.
Create the script
ntp.shon your cluster:
dbutils.fs.put("/databricks/init_scripts/ntp.sh",""" #!/bin/bash echo "<ntp-server-ip> <ntp-server-hostname>" >> /etc/hosts cp /dbfs/databricks/init_scripts/ntp.conf /etc/ sudo service ntp restart""",True)
Confirm that the script exists:
Click Clusters, click your cluster name, click Edit, click Advanced Options, click Init Scripts.
Select DBFS under Destination.
Enter the full path to
ntp.shand click Add.
Click Confirm and Restart. A confirmation dialog box appears. Click Confirm and wait for the cluster to restart.