Using VoltDB

Documentation

VoltDB Home » Documentation » Using VoltDB

Using VoltDB

V4.8

The text and illustrations in this document are licensed under the terms of the GNU Affero General Public License Version 3 as published by the Free Software Foundation. See the GNU Affero General Public License (http://www.gnu.org/licenses/) for more details.

Many of the core VoltDB database features described herein are part of the VoltDB Community Edition, which is licensed under the GNU Affero Public License 3 as published by the Free Software Foundation. Other features are specific to the VoltDB Enterprise Edition, which is distributed by VoltDB, Inc. under a commercial license. Your rights to access and use VoltDB features described herein are defined by the license you received when you acquired the software.

Abstract

This book explains how to use VoltDB to design, build, and run high performance applications.


Table of Contents

Preface
1. Overview
1.1. What is VoltDB?
1.2. Who Should Use VoltDB
1.3. How VoltDB Works
1.3.1. Partitioning
1.3.2. Serialized (Single-Threaded) Processing
1.3.3. Partitioned vs. Replicated Tables
1.3.4. Ease of Scaling to Meet Application Needs
2. Installing VoltDB
2.1. Operating System and Software Requirements
2.2. Installing VoltDB
2.2.1. Upgrading From Older Versions
2.2.2. Installing Standard System Packages
2.2.3. Building a New VoltDB Distribution Kit
2.3. Setting Up Your Environment
2.4. What is Included in the VoltDB Distribution
2.5. VoltDB in Action: Running the Sample Applications
3. Designing Your VoltDB Application
3.1. Designing the Database
3.1.1. Partitioning Database Tables
3.1.2. Replicating Tables
3.2. Designing the Data Access (Stored Procedures)
3.2.1. Writing VoltDB Stored Procedures
3.2.2. VoltDB Stored Procedures and Determinism
3.2.3. The Anatomy of a VoltDB Stored Procedure
3.2.4. Partitioning Stored Procedures
3.3. Designing the Application Logic
3.3.1. Connecting to the VoltDB Database
3.3.2. Invoking Stored Procedures
3.3.3. Invoking Stored Procedures Asynchronously
3.3.4. Closing the Connection
3.4. Handling Errors
3.4.1. Interpreting Execution Errors
3.4.2. Handling Timeouts
3.4.3. Interpreting Other Errors
4. Simplifying Application Development
4.1. Default Procedures
4.2. Shortcut for Defining Simple Stored Procedures
4.3. Writing Stored Procedures Inline Using Groovy
4.4. Verifying Expected Query Results
5. Building Your VoltDB Application
5.1. Compiling the Client Application and Stored Procedures
5.2. Declaring the Stored Procedures
5.3. Building the Application Catalog
6. Running Your VoltDB Application
6.1. Defining the Cluster Configuration
6.1.1. Determining How Many Partitions to Use
6.1.2. Configuring Paths for Runtime Features
6.1.3. Verifying your Hardware Configuration
6.2. Starting a VoltDB Database for the First Time
6.2.1. Simplifying Startup on a Cluster
6.2.2. How VoltDB Database Startup Works
6.3. Starting VoltDB Client Applications
6.4. Shutting Down a VoltDB Database
6.5. Stopping and Restarting a VoltDB Database
6.5.1. Save and Restore
6.5.2. Command Logging and Recovery
6.6. Modes of Operation
6.6.1. Admin Mode
6.6.2. Starting the Database in Admin Mode
7. Updating Your VoltDB Database
7.1. Planning Your Application Updates
7.2. Updating the Database Schema on a Running Database
7.2.1. Validating the Updated Catalog
7.2.2. Managing the Update Process
7.3. Updating the Database Using Save and Restore
7.4. Updating the Hardware Configuration
7.4.1. Adding Nodes with Elastic Scaling
7.4.2. Configuring How VoltDB Rebalances New Nodes
8. Security
8.1. How Security Works in VoltDB
8.2. Enabling Authentication and Authorization
8.3. Defining Users and Roles
8.4. Assigning Access to Stored Procedures
8.5. Allowing Access to System Procedures, Ad Hoc Queries, and Default Procedures
8.6. Integrating Kerberos Security with VoltDB
8.6.1. Installing and Configuring Kerberos
8.6.2. Installing and Configuring the JAVA Security Extensions
8.6.3. Configuring the VoltDB Servers and Clients
9. Saving & Restoring a VoltDB Database
9.1. Performing a Manual Save and Restore of a VoltDB Cluster
9.1.1. How to Save the Contents of a VoltDB Database
9.1.2. How to Restore the Contents of a VoltDB Database
9.1.3. Changing the Database Schema or Cluster Configuration Using Save and Restore
9.2. Scheduling Automated Snapshots
9.3. Managing Snapshots
9.4. Special Notes Concerning Save and Restore
10. Command Logging and Recovery
10.1. How Command Logging Works
10.2. Controlling Command Logging
10.3. Configuring Command Logging for Optimal Performance
10.3.1. Log Size
10.3.2. Log Frequency
10.3.3. Synchronous vs. Asynchronous Logging
10.3.4. Hardware Considerations
11. Availability
11.1. How K-Safety Works
11.2. Enabling K-Safety
11.2.1. What Happens When You Enable K-Safety
11.2.2. Calculating the Appropriate Number of Nodes for K-Safety
11.3. Recovering from System Failures
11.3.1. What Happens When a Node Rejoins the Cluster
11.3.2. Where and When Recovery May Fail
11.4. Avoiding Network Partitions
11.4.1. K-Safety and Network Partitions
11.4.2. Using Network Fault Protection
12. Database Replication
12.1. How Database Replication Works
12.1.1. Starting Replication
12.1.2. Replication and Existing Databases
12.1.3. Database Replication and Disaster Recovery
12.1.4. Database Replication and Completeness
12.1.5. Database Replication and Read-only Clients
12.2. Database Replication in Action
12.2.1. Starting Replication
12.2.2. Stopping Replication
12.2.3. Promoting the Replica When the Master Becomes Unavailable
12.2.4. Managing Database Replication
12.3. Using the Sample Applications to Demonstrate Replication
12.3.1. Replicating the Voter Sample Using the Enterprise Manager
12.3.2. Replicating the Voter Sample Using the Command Line
13. Exporting Live Data
13.1. Understanding Export
13.2. Planning your Export Strategy
13.3. Identifying Export Tables in the Schema
13.4. Configuring Export in the Deployment File
13.5. The File Connector
13.6. The HTTP Connector
13.7. The JDBC Connector
13.8. The Kafka Connector
13.9. The RabbitMQ Connector
13.10. How Export Works
13.10.1. Export Overflow
13.10.2. Persistence Across Database Sessions
14. Logging and Analyzing Activity in a VoltDB Database
14.1. Introduction to Logging
14.2. Creating the Logging Configuration File
14.3. Enabling Logging for VoltDB
14.4. Customizing Logging in the VoltDB Enterprise Manager
14.5. Changing the Timezone of Log Messages
14.6. Changing the Configuration on the Fly
15. Using VoltDB with Other Programming Languages
15.1. C++ Client Interface
15.1.1. Writing VoltDB Client Applications in C++
15.1.2. Creating a Connection to the Database Cluster
15.1.3. Invoking Stored Procedures
15.1.4. Invoking Stored Procedures Asynchronously
15.1.5. Interpreting the Results
15.2. JSON HTTP Interface
15.2.1. How the JSON Interface Works
15.2.2. Using the JSON Interface from Client Applications
15.2.3. How Parameters Are Interpreted
15.2.4. Interpreting the JSON Results
15.2.5. Error Handling using the JSON Interface
15.3. JDBC Interface
15.3.1. Using JDBC to Connect to a VoltDB Database
15.3.2. Using JDBC to Query a VoltDB Database
A. Supported SQL DDL Statements
CREATE INDEX — Creates an index for faster access to a table.
CREATE PROCEDURE AS — Defines a stored procedure composed of a SQL query.
CREATE PROCEDURE FROM CLASS — Defines a stored procedure associated with a Java class.
CREATE ROLE — Defines a role and the permissions associated with that role.
CREATE TABLE — Creates a table in the database.
CREATE VIEW — Creates a view into a table, used to optimize access to specific columns within a table.
EXPORT TABLE — Specifies that a table is for export only.
IMPORT CLASS — Specifies additional Java classes to include in the application catalog.
PARTITION PROCEDURE — Specifies that a stored procedure is partitioned.
PARTITION TABLE — Specifies that a table is partitioned and which is the partitioning column.
B. Supported SQL Statements
DELETE — Deletes one or more records from the database.
INSERT — Creates new rows in the database, using the specified values for the columns.
SELECT — Fetches the specified rows and columns from the database.
TRUNCATE TABLE — Deletes all records from the specified table.
UPDATE — Updates the values within the specified columns and rows of the database.
UPSERT — Either inserts new rows or updates existing rows depending on the primary key value.
C. SQL Functions
ABS() — Returns the absolute value of a numeric expression.
ARRAY_ELEMENT() — Returns the element at the specified location in a JSON array.
ARRAY_LENGTH() — Returns the number of elements in a JSON array.
AVG() — Returns the average of a range of numeric column values.
CAST() — Explicitly converts an expression to the specified datatype.
CEILING() — Returns the smallest integer value greater than or equal to a numeric expression.
CHAR() — Returns a string with a single UTF-8 character associated with the specified character code.
CHAR_LENGTH() — Returns the number of characters in a string.
COALESCE() — Returns the first non-null argument, or null.
CONCAT() — Concatenates two or more strings and returns the result.
COUNT() — Returns the number of rows selected containing the specified column.
CURRENT_TIMESTAMP — Returns the current time as a timestamp value.
DAY(), DAYOFMONTH() — Returns the day of the month as an integer value.
DAYOFWEEK() — Returns the day of the week as an integer between 1 and 7.
DAYOFYEAR() — Returns the day of the year as an integer between 1 and 366.
DECODE() — Evaluates an expression against one or more alternatives and returns the matching response.
EXP() — Returns the exponential of the specified numeric expression.
EXTRACT() — Returns the value of a selected portion of a timestamp.
FIELD() — Extracts a field value from a JSON-encoded string column.
FLOOR() — Returns the largest integer value less than or equal to a numeric expression.
FORMAT_CURRENCY() — Converts a DECIMAL to a text string as a monetary value.
FROM_UNIXTIME() — Converts a UNIX time value to a VoltDB timestamp.
HOUR() — Returns the hour of the day as an integer value.
LEFT() — Returns a substring from the beginning of a string.
LOWER() — Returns a string converted to all lowercase characters.
MAX() — Returns the maximum value from a range of column values.
MIN() — Returns the minimum value from a range of column values.
MINUTE() — Returns the minute of the hour as an integer value.
MONTH() — Returns the month of the year as an integer value.
NOW — Returns the current time as a timestamp value.
OCTET_LENGTH() — Returns the number of bytes in a string.
OVERLAY() — Returns a string overwriting a portion of the original string with the specified replacement.
POSITION() — Returns the starting position of a substring in another string.
POWER() — Returns the value of the first argument raised to the power of the second argument.
QUARTER() — Returns the quarter of the year as an integer value
REPEAT() — Returns a string composed of a substring repeated the specified number of times.
REPLACE() — Returns a string replacing the specified substring of the original string with new text.
RIGHT() — Returns a substring from the end of a string.
SECOND() — Returns the seconds of the minute as a floating point value.
SET_FIELD() — Returns a copy of a JSON-encoded string, replacing the specified field value.
SINCE_EPOCH() — Converts a VoltDB timestamp to an integer number of time units since the POSIX epoch.
SPACE() — Returns a string of spaces of the specified length.
SQRT() — Returns the square root of a numeric expression.
SUBSTRING() — Returns the specified portion of a string expression.
SUM() — Returns the sum of a range of numeric column values.
TO_TIMESTAMP() — Converts an integer value to a VoltDB timestamp based on the time unit specified.
TRIM() — Returns a string with leading and/or training spaces removed.
TRUNCATE() — Truncates a VoltDB timestamp to the specified time unit.
UPPER() — Returns a string converted to all uppercase characters.
WEEK(), WEEKOFYEAR() — Returns the week of the year as an integer value.
WEEKDAY() — Returns the day of the week as an integer between 0 and 6.
YEAR() — Returns the year as an integer value.
D. VoltDB CLI Commands
csvloader — Imports the contents of a CSV file and inserts it into a VoltDB table.
dragent — Starts the database replication agent.
jdbcloader — Extracts a table from another database via JDBC and inserts it into a VoltDB table.
kafkaloader — Imports data from a Kafka message queue into the specified database table.
sqlcmd — Starts an interactive command prompt for issuing SQL queries to a running VoltDB database
voltadmin — Performs administrative functions on a VoltDB database.
voltdb — Performs management tasks on the current server, such as compiling the application catalog and starting the database.
E. Deployment File (deployment.xml)
E.1. Understanding XML Syntax
E.2. The Structure of the Deployment File
F. VoltDB Datatype Compatibility
F.1. Java and VoltDB Datatype Compatibility
G. System Procedures
@AdHoc — Executes an SQL statement specified at runtime.
@Explain — Returns the execution plan for the specified SQL query.
@ExplainProc — Returns the execution plans for all SQL queries in the specified stored procedure.
@GetPartitionKeys — Returns a list of partition values, one for every partition in the database.
@Pause — Initiates admin mode on the cluster.
@Promote — Promotes a replica database to normal operation.
@Quiesce — Waits for all queued export data to be written to the connector.
@Resume — Returns a paused database to normal operating mode.
@Shutdown — Shuts down the database.
@SnapshotDelete — Deletes one or more native snapshots.
@SnapshotRestore — Restores a database from disk using a native format snapshot.
@SnapshotSave — Saves the current database contents to disk.
@SnapshotScan — Lists information about existing native snapshots in a given directory path.
@SnapshotStatus — Lists information about the most recent snapshots created from the current database.
@Statistics — Returns statistics about the usage of the VoltDB database.
@StopNode — Stops a VoltDB server process, removing the node from the cluster.
@SystemCatalog — Returns metadata about the database schema.
@SystemInformation — Returns configuration information about VoltDB and the individual nodes of the database cluster.
@UpdateApplicationCatalog — Reconfigures the database by replacing the application catalog currently in use.
@UpdateLogging — Changes the logging configuration for a running database.
>