Archive
Archive
Before you start the cluster, you choose what database features to use by specifying a configuration file when you initialize the database root directory on each node using the voltdb init command. You must specify the same configuration file on every node of the cluster. For example:
$ voltdb init --config=deployment.xml
If you choose to change database options, many of the features can be adjusted while the database is running by either:
Using the web-based VoltDB Management Center to make changes interactively in the Admin tab
Editing the original configuration file and applying the modifications with the voltadmin update command
For example, you can change security settings, import and export configurations, and resource limits dynamically. With either approach, the changes you make are saved by VoltDB in the database root directory.
However, there are some changes that cannot be made while the database is running. For example, changing the K-safety value or the number of partitions per server require shutting down, re-initializing, and restarting the database. To change these static aspects of your cluster, you must save the database contents, reconfigure the root directory, then restart and restore the database. The steps for changing static configuration options are:
Pause the database (voltadmin pause)
Save a snapshot of the contents (voltadmin save {path} {file-prefix})
Shutdown the database (voltadmin shutdown)
Re-initialize the root directory with the new configuration file and the --force argument
(voltdb init --force --config=file)
Restart the database in admin mode (voltdb start --pause)
Restore the snapshot (voltadmin restore {path} {file-prefix})
Resume normal operations (voltadmin resume)
See Chapter 13, Saving & Restoring a VoltDB Database for information on using save and restore. When doing benchmarking, where you need to change the number of partitions, reducing the number of servers, or adjusting other static configuration options, this is the recommended approach. However, if you are simply adding nodes to the cluster to add capacity or increase performance, you can add the nodes while the database is running. Adding nodes "on the fly" is also known as elastic scaling, as described in the next section.
When you are ready to extend the cluster by adding one or more nodes, you simply initialize and start the VoltDB
database process on the new nodes using the voltdb init command to initialize and the voltdb
start command to start with the --add argument, specifying the name of one or more of the
existing cluster nodes as the hosts. For example, if you are adding node ServerX to a cluster where ServerA is already a
member, you can execute the following commands on ServerX:
$ voltdb init --config=deployment.xml $ voltdb start --add --host=ServerA
Once the elastic add action is initiated, the cluster performs the following tasks:
The cluster acknowledges the presence of a new server.
Copies of the current schema and configuration settings are sent to the new node.
Once sufficient nodes are added, copies of all replicated tables and their share of the partitioned tables are sent to the new nodes.
As the data is redistributed (or rebalanced), the added nodes begin participating as full members of the cluster.
There are some important notes to consider when expanding the cluster using elastic scaling:
You must add a sufficient number of nodes to create an integral K-safe unit. That is, K+1 nodes. For example, if the K-safety value for the cluster is two, you must add three nodes at a time to expand the cluster. If the cluster is not K-safe (in other words it has a K-safety value of zero), you can add one node at a time.
When you add nodes to a K-safe cluster, the nodes added first will complete steps #1 and #2 above, but will not complete steps #3 and #4 until the correct number of nodes are added, at which point all nodes rebalance together.
While the cluster is rebalancing (Step #3), the database continues to handle incoming requests. However, depending on the workload and amount of data in the database, rebalancing may take a significant amount of time.
Once elastic scaling is complete, your database configuration has changed. If you shutdown the database and then
restart, you must specify the new server count in the --count argument to the voltdb
start command.
Once you add the necessary number of nodes (based on the K-safety value), VoltDB rebalances the cluster, moving data from existing partitions to the new nodes. During the rebalance operation, the database remains available and actively processing client requests. How long the rebalance operation takes is dependent on two factors: how often rebalance tasks are processed and how much data each transaction moves.
Rebalance tasks are fully transactional, meaning they operate within the database's ACID-compliant transactional model. Because they involve moving data between two or more partitions, they are also multi-partition transactions. This means that each rebalance work unit can incrementally add to the latency of pending client transactions.
You can control how quickly the rebalance operation completes versus how much rebalance work impacts ongoing client
transactions using two attributes of the <elastic> element in the configuration
file:
The duration attribute sets a target value for the length of time each rebalance transaction will take, specified in milliseconds. The default is 50 milliseconds.
The throughput attribute sets a target value for the number of megabytes per second that will be processed by the rebalance transactions. The default is 2 megabytes.
When you change the target duration, VoltDB adjusts the amount of data that is moved in each transaction to reach the target execution time. If you increase the duration, the volume of data moved per transaction increases. Similarly, if you reduce the duration, the volume per transaction decreases.
When you change the target throughput, VoltDB adjusts the frequency of rebalance transactions to achieve the desired volume of data moved per second. If you increase the target throughout, the number of rebalance transactions per second increases. Similarly, if you decrease the target throughout, the number of transactions decreases.
The <elastic> element is a child of the <systemsettings> element. For example, the following configuration file sets the target duration to 15 milliseconds and the target throughput to 1 megabyte per second before starting the database:
V8.x Documentation<deployment>
. . .
<systemsettings>
<elastic duration="15" throughput="1"/>
</systemsettings>
</deployment>