5.2. Updating the Database Configuration

Documentation

VoltDB Home » Documentation » VoltDB Kubernetes Administrator's Guide

5.2. Updating the Database Configuration

You can also change the configuration options for the database or the cluster while the database is running. In Kubernetes, you do this by updating the release properties rather than with the voltadmin update command.

How you update the configuration properties is the same for all properties: you use the helm upgrade command to update the individual properties. However, what actions result from the update depend on the type of properties you want to modify:

  • Dynamic database configuration properties that can be modified "on the fly" without restarting the database

  • Static database configuration properties that require the database be restarted before they are applied

  • Cluster configuration properties that alter the operation of the cluster and associated Kubernetes pods

The following sections describe these three circumstances in detail.

5.2.1. Changing Database Properties on the Running Database

There are a number of database configuration options that can be changed while the database is running. Those options include:

  • Security settings, including user accounts

    cluster.config.deployment.security.enabled
    cluster.config.deployment.users
  • Import and export settings

    cluster.config.deployment.export.configurations
    cluster.config.deployment.import.configurations
  • Database replication settings (except the DR cluster ID)

    cluster.config.deployment.dr.role
    cluster.config.deployment.dr.connection
  • Automated snapshots

    cluster.config.deployment.snapshot.*
  • Heartbeat timeout

    cluster.config.deployment.heartbeat.timeout
  • System settings:

    cluster.config.deployment.systemsettings.compaction.*
    cluster.config.deployment.systemsettings.elastic.*
    cluster.config.deployment.systemsettings.flushinterval.*
    cluster.config.deployment.systemsettings.query.timeout
    cluster.config.deployment.systemsettings.resourcemonitor.*

For example, the following helm upgrade command changes the heartbeat timeout to 30 seconds:

$ helm upgrade mydb voltdb/voltdb       \
   --reuse-values                       \
   --set cluster.config.deployment.heartbeat.timeout=30

When dynamic configuration properties are modified, the VoltDB Operator updates the running database configuration as soon as it is notified of the change.

5.2.2. Changing Database Properties That Require a Restart

Many database configuration properties are static — they cannot be changed without restarting the database. Normally, this requires manually performing a voltadmin shutdown --save, reinitializing and restarting the database cluster, then restoring the final snapshot. For example, command logging cannot be turned on or off while the database is running; similarly, the number of sites per host cannot be altered on the fly.

However, you can change these properties using the helm upgrade command and the VoltDB Operator will make the changes, but not while the database is running. Instead, the Operator recognizes the changes to the configuration, marks the database as requiring a restart, and then schedules a shutdown snapshot, reinitialization, and restart of the database for later.

For example, you cannot change the number of sites per host while the database is running. But the Operator does let you change the property in Kubernetes:

$ helm upgrade mydb voltdb/voltdb       \
   --reuse-values                       \
   --set cluster.config.deployment.cluster.siteperhost=12

No action is taken immediately, since the change will require a restart and is likely to interrupt ongoing transactions. Instead, the Operator waits until you are ready to restart the cluster, which you signify by changing another property, cluster.clusterSpec.allowRestartDuringUpdate, to true:

$ helm upgrade mydb voltdb/voltdb       \
   --reuse-values                       \
   --set cluster.clusterSpec.allowRestartDuringUpdate=true

Important

You should never set the .allowResstartDuringUpdate property to true for an XDCR cluster. If you do, the cluster will not be able to rejoin the XDCR mesh and will fail during startup. To change configuration options that require a restart for an XDCR cluster, you must:

If you are sure you are ready to restart the cluster when you change the configuration property, you can set the two properties at the same time so that the change takes immediate effect:

$ helm upgrade mydb voltdb/voltdb                  \
   --reuse-values                                  \
   --set cluster.config.deployment.cluster.siteperhost=12  \
   --set cluster.clusterSpec.allowRestartDuringUpdate=true

Once allowRestartDuringUpdate is set to true, the Operator initiates the restart process, saving, shutting down, reinitializing, restarting and restoring the database automatically. Note that once the database is restarted, it is a good idea to reset allowRestartDuringUpdate to false to avoid future configuration changes triggering immediate restarts:

$ helm upgrade mydb voltdb/voltdb       \
   --reuse-values                       \
   --set cluster.clusterSpec.allowRestartDuringUpdate=false

Warning

There are certain database configuration changes that cannot be made either on the fly or with a restart. In particular, do not attempt to change properties associated with directory paths or SSL configuration. Changing any of these properties will leave your database in an unstable state.

5.2.3. Changing Cluster Properties

There are properties associated with the environment that the VoltDB database runs on that you can also modify with the helm upgrade command. Most notably, you can increase the size of the cluster, using elastic scaling, by changing the cluster.clusterSpec.replicas property, as described in Section 3.3, “Resizing the Cluster with Elastic Scaling”.

Some properties affect the computing environment, such as environment variables and number of nodes. Others control the network ports assigned or features specific to Kubernetes, such as liveness and readiness. All these properties can be modified. However, they each have separate scopes that affect when the changes will go into affect.

Of particular note, pod-specific properties will not take affect until each pod restarts. If this is not a high availability cluster (that is, K=0), the Operator will wait until you to change the property cluster.clusterSpec.allowRestartDuringUpdate to true before restarting the cluster and applying the changes. The same applies for any cluster-wide properties.

However, for a K-safe cluster, the Operator can apply pod-specific changes without any downtime by performing a rolling upgrade. That is, stopping and replacing each pod in sequence. So for high availability clusters, the Operator will start applying pod-specific changes automatically via a rolling restart regardless of the cluster.clusterSpec.allowRestartDuringUpdate setting.