Hardening VoltDB Security

This document explains how to harden your VoltDB database and applications against a variety of common security threats, both malicious and accidental. The following table summarizes the recommendations, which are explained in more detail below.

In Brief:

Harden The Environment

  1. Set up a system-level firewall.

  2. Keep system software up to date.

  3. Use disk encryption to protect VoltDB data and configuration.

  4. Limit system user permissions.

Monitor

  1. Look for failed or unusual authentication messages in the VoltDB log.

  2. Look for unusual or unauthorized behavior at the system level, such as port scans, failed logins, strange network activity, etc...

Malicious vs. Accidental Attacks

When we talk about hardening VoltDB, we are usually talking about protecting the system from malicious attackers, trying to steal, manipulate, ransom, disable or or embarrass.

It is also prudent to harden against accidental misuse. Much like avoiding logging in as root to your servers, limiting access to internal users and systems is a best practice as well.

Environmental Security

Sensible Firewall Defaults

Your server should have a firewall enabled and should limit outside access to only the ports needed. If whole-internet access is not needed on open ports, incoming connections can be limited to certain IPs, ranges or subnets.

Encryption for Data at Rest

While billed as an in-memory database, VoltDB stores configuration and data on disks. Both Linux and MacOS support various kinds of disk encryption to increase the security of data at rest. Enabling this protection usually has a very small effect on throughput, and can protect you from certain kinds of attacks:

File encryption should be used for backup, whether automated or manual. Backups are often an easier target than production servers, and read access may not be under the VoltDB administrator's control.

Encryption between Clusters

The most secure way to connect two VoltDB clusters is over a secure tunnel. There are many ways to set up such a tunnel, but the important thing is that all data is encrypted on the wire over the public internet.

Limiting Attack Surface Area

Ports

VoltDB listens on a number of ports by default. All of them can be set to listen on different ports.

Note, changing the ports from their default values does not make VoltDB more secure or limit surface area for targeted attacks. It does have the advantage of making your VoltDB servers less of a target. It makes port scanners that cast a wide net looking for exposed vulnerable servers work much harder.

For example, the default client port is 21212. It isn't very hard to scan every IPv4 address looking for those listening on 21212 and check if they are an exposed VoltDB server. It's much harder to scan thousands of ports on every IPv4 address looking for exposed VoltDB servers.

https://docs.voltdb.com/AdminGuide/HostConfigPortOpts.php

Interfaces

Most servers have at least two interfaces they can use for network access, a loopback interface (localhost, 127.0.0.1) and one that has broader internet access. Many servers have additional interfaces for internal-only networks or VPNs. More and more systems have IPv4 and IPv6 interfaces.

VoltDB allows you to instruct it to listen on specific interfaces. If clients are running on an internal network, set the VoltDB client interface to listen only on that internal network. This will make it much harder for outside attackers to gain access to VoltDB through the client port.

Note: it's possible to have VoltDB's admin port listen on a more restricted interface than the public client interface port.

https://docs.voltdb.com/UsingVoltDB/clivoltdb.php

Firewalls

As mentioned above, firewalls can be used to limit incoming (or outgoing) connections to only the most narrow ports and networks needed.

Monitoring and Administrative Tools

Tools that connect to VoltDB should use the most restricted network possible. They should use their own credentials. Monitoring tools should not have write permissions.

Understanding Pause Mode

Putting a VoltDB server in pause mode disables any transaction that may write to state on the client port (and interface if specified). It does not affect the second, administrative port. It is designed to make VoltDB operations simpler, and is typically not an effective tool for security.

Limiting Client Permissions

Each piece of software that connects to VoltDB should use a role that allows it to do what it needs to do and no more. For example, monitoring systems should be read only and client applications should not have administrative privileges. Furthermore, each piece of software should have unique credentials. For example, four app servers may share one user/password, but monitoring software should use a different user/password.

This limits the damage if these client applications are compromised and allows you to quickly disable access to individual services. It also adds some protection from internal accidental misuse.

Authentication & Access Control

Enabling Security

The single most effective thing you can do to harden VoltDB is turn on user and role-based security. Today, security is disabled by default to enable downloaders to experiment with VoltDB with less friction. For any sensitive data or application logic, security should be enabled.

Note that a well-planned authentication and access control system is necessary, but not sufficient to harden VoltDB.

https://docs.voltdb.com/UsingVoltDB/ChapSecurity.php#SecurityHowItWorks

https://docs.voltdb.com/UsingVoltDB/SecurityEnable.php

Users, Permissions & Roles

A VoltDB user has a username and password and is assigned to one or more roles. VoltDB can have zero or more users configured in the deployment file.

A permission (or privilege) is the right to do something in the VoltDB system. There are some special built-in privileges, like ADMIN, SQL, ALLPROC, etc.. Note that default permissions like SQLREAD are more restrictive than SQL, and can restrict users to read-only actions. There are also permissions for individual stored procedures.

A role is a collection of permissions. There are two default roles. Administrator has full permissions. User has access to all non-administrative SQL and procedures. You can define additional roles in your DDL that have different permissions than the built-in role, User.

Using these tools to limit the access of clients to precisely what they need to function will make your system more secure.

https://docs.voltdb.com/UsingVoltDB/SecurityUsersGroups.php

https://docs.voltdb.com/UsingVoltDB/SecuritySPs.php

https://docs.voltdb.com/UsingVoltDB/SecuritySysprocs.php

https://docs.voltdb.com/UsingVoltDB/SecurityDefaultRoles.php

Procedure-Based Access Control vs Table-Based Access Control

Many database systems feature permissions by table, not by procedure. VoltDB's fine-grained permissions are by procedure. Many VoltDB clients interact with VoltDB through procedures. Those clients that do use SQL can often get by with full SQL or read-only SQL permissions. This is something to be aware of when planning roles and permissions.

Keeping Passwords Safe

Clearly an attacker accessing your VoltDB username and password is a bad thing.

Between client and server, passwords are hashed before they are sent across the wire. You can also use pre-hashed passwords directly in your client code. Hashed passwords protect the plaintext password string. Note that hashing the password in this way doesn't limit access to VoltDB for eavesdroppers of data transmitted on the wire. Credential data on the wire must be protected from attackers, using either internal-only networks, encrypted tunnels or Kerberos (described in the next section).

In the deployment file, the "voltdb mask" command can be used to store hashed passwords in the deployment file, which also protects a user's plaintext password.

https://docs.voltdb.com/UsingVoltDB/clivoltdb.php

Note that even when using masked passwords, if an attacker gains access to a deployment file containing a list of usernames and passwords, it will be vastly easier to log into VoltDB. This file should be protected with system level security, including Unix permissions, disk encryption and other typical security measures for secrets stored on disk.

Standard vs Kerberos Authentication

A more involved, but more secure (if implemented with best practices) way of protecting credentials is using Kerberos. VoltDB natively supports using Kerberos for authentication credentials.

https://docs.voltdb.com/UsingVoltDB/SecurityKerberos.php

Differentiating Production and Pre-Production Clusters

It is a good idea to make it hard to accidentally point a development client at a production cluster. This happens surprisingly often. One way to do that is to use different credentials for production, pre-production, and development clusters. Additionally, don't make the clients auto-switch credentials depending on connection targets.

There are many other ways to make this harder to do accidentally as well. This is just one more thing to consider when hardening systems.

Auditing Access and Behavior

VoltDB Logging and Monitoring

By default, VoltDB logs authentication attempts, whether successful or not. Specifically, the authentication system logs to the AUTH logger at the INFO level. Note that successful authentication log messages are rate limited to once per minute. Failed attempts are logged individually.

Operators should monitor failed and successful authentication attempts and verify they follow expected patterns.