Chapter 6. Monitoring VoltDB Databases in Kubernetes

Documentation

VoltDB Home » Documentation » VoltDB Kubernetes Administrator's Guide
< previoustable of contentsnext >

Chapter 6. Monitoring VoltDB Databases in Kubernetes

Once the database is running, you need to monitor the system to ensure reliable uptime and performance. Variations in usage, workload, or the operational environment can affect the dynamics of the data application, which may need corresponding adjustments to the schema, procedures, or hardware configuration. VoltDB provides system procedures (such as @Statistics) and the web-based Volt Management Center to help monitor current performance. But to provide persistent, historical intelligence concerning application performance it is best to use a dedicated metrics data store, such as Prometheus.

Prometheus is a metrics monitoring and alerting system that provides ongoing data collection and persistent storage for applications and other resources. By providing an open source industry standard for collecting and storing metrics, Prometheus allows you to:

  • Offload monitoring from the database platform itself

  • Combine metrics from VoltDB with other applications within your business ecosystem

  • Query and visualize historical information about your database activity and performance (through tools such as Grafana)

Section 6.1, “Using Prometheus to Monitor VoltDB” explains how to configure your VoltDB database so the information you need is gathered and made available through Prometheus and compatible graphic consoles such as Grafana.

6.1. Using Prometheus to Monitor VoltDB

When monitoring VoltDB with Prometheus on Kubernetes, the Operator starts an auxiliary pod to run the Prometheus agent, with one agent providing metrics for the entire cluster. The agent takes data from the database (through the system procedures @Statistics and @SystemInformation) and makes them available as metrics in the format that Prometheus requires. In other words, the agent stands between the VoltDB cluster and the Prometheus server, doing the data collection and translation needed.

Before you start, you must make sure Prometheus is installed on your Kubernetes cluster, ideally within the same namespace as your VoltDB instance. Once the Prometheus stack is available, you can enable and configure the Prometheus agent using the metrics... properties in Helm. First, set the metrics.enabled property to true. You can set this property when initially configuring the database or using helm upgrade after the database is running. For example, the following command enables the Prometheus agent with default settings while initializing the mydb database cluster:

$ helm install mydb voltdb/voltdb                        \
   --set-file cluster.config.licenseXMLFile=license.xml  \
   --set cluster.clusterSpec.replicas=5                  \
   --set metrics.enabled=true

Once enabled, the agent starts in a separate pod using the database name (in this case "mydb"), the string "voltdb-metrics", and an arbitrary suffix as the pod name. The agent then uses port number 8484 to report its metrics. By default, the connection to the Prometheus server is handled automatically so you do not need to add the agent as a target manually to the Prometheus configuration. Unless you wish to report your metrics to a Prometheus server outside the current Kubernetes environment.

If the database has security enabled, you will also need to set the metrics.username and metrics.password properties specifying a valid account on the database cluster so the agent can access the system procedures. Similarly, if the database cluster is configured to use SSL/TLS, you must configure the Prometheus agent to use SSL/TLS as well, using the SSL properties such as metrics.ssl.enabled and metrics.ssl.truststore.file, making sure the agent uses a truststore that can validate against the database's certificate. For example, the following Helm command enables SSL for both the database and the Prometheus agent, using the same user-generated certificate for both:

$ helm install mydb voltdb/voltdb                                        \
   --set-file cluster.config.licenseXMLFile=license.xml                  \
   --set cluster.clusterSpec.replicas=5                                  \
   --set cluster.config.deployment.ssl.enabled=true                      \
   --set cluster.config.deployment.ssl.keystore.password=topsecret       \
   --set cluster.config.deployment.ssl.truststore.password=topsecret     \
   --set-file cluster.config.deployment.ssl.keystore.file=mycert.jks     \
   --set-file cluster.config.deployment.ssl.truststore.file=mytrust.jks  \
   --set-file cluster.clusterSpec.ssl.certificateFile=mytrust.pem        \
   --set metrics.enabled=true                                            \
   --set metrics.ssl.enabled=true                                        \
   --set metrics.ssl.truststore.password=topsecret                       \
   --set-file metrics.ssl.truststore.file=mytrust.jks

Additional properties are available for configuring other aspects of the agent, such as filtering which statistics or system information to include as metrics. For example, the following YAML file enables the Prometheus agent and excludes the import and task statistics:

metrics:
    enabled: true
    skipstats: import,task

See Section B.8, “Metrics Configuration Options” for a full list of metrics properties.

< previoustable of contentsnext >