The preceding recommendations for using NTP work equally well in a managed or a hosted environment. However, there are some additional issues that can arise when working in a hosted environment that should be considered.
In a locally managed environment, you have complete control over both the hardware and software configuration. This means you can ensure that the VoltDB cluster nodes are connected to the same switch and in close proximity to a private NTP server, guaranteeing the best network performance within the cluster and to the NTP reference server.
In a hosted environment, you may not have control over the physical arrangement of servers but you usually have control of the software configuration.
In a virtualized or cloud environment, you have no control over — or even knowledge of — the hardware configuration. You are often using a predefined system image or "instance", including the operating system and time management configuration, which may not be appropriate for VoltDB. There are configuration changes you should consider making each time you "spin up" a new virtual server.
In situations where you have control over the selection and configuration of the server operating system and services, the preceding recommendations for configuring NTP should be sufficient. The key concern would be those aspects of the environment you do not have control over: network bandwidth and reliability. Again, the recommended NTP configuration in Section 14.2.4, “Example NTP Configuration”, especially the use of a local timer server and peer relationship within the cluster, should provide reliable time management despite any network fluctuations.
In virtual or cloud environments, you usually do not have control over either the hardware or the initial software configuration. New servers are instantiated from a common system image, or "instance", with default configurations for the operating system and time management. This presents two problems for establishing a reliable environment for VoltDB:
The default configuration may not be sufficient and must be overridden
Because of the prior issue, there can be considerable clock skew that must be corrected before running VoltDB
Virtualization allows multiple virtual servers to run on a single piece of hardware. To do this, prepackaged "instances" of an operating system are booted under a virtual machine manager. These instances are designed to support the majority of applications, most of which do not have extensive requirements for clock synchronization, As a result, the instances often use default NTP configurations or none at all.
When you spin up a new virtual server, in most cases you need to reconfigure NTP, changing the configuration file as described in Section 14.2.4, “Example NTP Configuration” and restarting the service.
In some cases, NTP is not used at all. Instead, the operating system synchronizes its (virtual) clock against the
clock of the physical server on which it runs. You need to override this setting before installing, configuring, and
starting NTP. For example, when running early instances of Ubuntu in EC2 under the Xen hypervisor, you must modify the
file /proc/sys/xen/independent_wallclock to avoid the hypervisor performing the clock
synchronization. For example:
$ echo "1" > /proc/sys/xen/independent_wallclock $ apt-get install -y ntp
This particular approach is specific to the Xen hypervisor. Other virtualization engines may use a different approach for controlling the system clock. See the documentation for your specific virtualization environment for details.
Once NTP is running and managing the system clock, it can take a considerable amount of time for the clocks to
synchronize if the initial skew is large. You can reduce this initial delay by forcing synchronization before you start
VoltDB. You can do this performing the following steps as the user root:
Stop the NTP service.
Use the ntpdate command to synchronize against a specific reference server. Do this several
times until the reported skew is consistently low. (It will never effectively be less than a millisecond — a
thousandth of a second — but can be reduced to a few milliseconds.)
Restart the NTP Service.
For example, if your local time server's IP address is 10.10.56.1, the commands might look like this:
$ service ntp stop * Stopping NTP server ntpd [ OK ] $ ntpdate -p 8 10.10.56.1 20 Oct 09:21:04 ntpdate[2795]: adjust time server 10.10.56.1 offset 0.008294 sec $ ntpdate -p 8 10.10.56.1 20 Oct 09:21:08 ntpdate[2797]: adjust time server 10.10.56.1 offset 0.002518 sec $ ntpdate -p 8 10.10.56.1 20 Oct 09:21:12 ntpdate[2798]: adjust time server 10.10.56.1 offset 0.001459 sec $ service ntp start -x * Starting NTP server ntpd [ OK ]
Once NTP is configured and the skew between the individual clocks and the reference server has been minimized, you can safely start the VoltDB database.
Documentation