Voltdb-table

This sink is suitable for efficiently inserting large volumes of data into VoltDB.

The voltdb-table sink is used to insert data into a VoltDB table in batches. It supports configurations such as batch size, flush interval, table name, and the type of bulk operation (INSERT or UPSERT).

When configuring servers to connect to it is enough to provide just one server and VoltDB client will discover the whole cluster topology, including partition assignment. In yaml configuration the servers field can be provided as a comma separated string or a list of individual host:port entries.

JAVAYAML

BulkInsertVoltSinkConfigBuilder.builder()
     .withTableName("my_table")
     .withBatchSize(100000)
     .withFlushInterval(5000)
     .withOperationType(VoltBulkOperationType.INSERT);

sink:
   voltdb-table:
       servers:
         - localhost:12122
       retries: 3
       name: "my_table"
       batchSize: 100000
       flushInterval: 5000
       operationType: "INSERT"

       client:
         maxTransactionsPerSecond: 3000
         maxOutstandingTransactions: 3000
         requestTimeout: PT10S
         auth:
           user: Admin
           password: 2r2Ffafw3V
         trustStore:
           file: file.pem
           password: got2have

Properties

servers

A set of host and port addresses for connecting to the VoltDB cluster. Only one address is sufficient for full cluster topology discovery. Required.

Type: array

tableName

The name of the VoltDB table where data will be inserted. Required.

Type: string

operationType

The type of bulk operation to perform, either INSERT or UPSERT. INSERT adds new records, while UPSERT updates existing records or inserts new ones. Type: object

Supported values: insert, upsert.

Default value: INSERT

batchSize

The maximum number of records to include in a single batch for insertion. Larger batches can improve performance but require more memory. Type: number

Default value: 100000

flushInterval

The maximum time to wait for the desired batch size before forcing a data flush to VoltDB. Type: object

Default value: 1s

client

Configuration settings for the VoltDB client, including retry policies, transaction limits, and authentication details. Type: object

Fields of client:

client.retry

Configuration for retrying failed operations, including the number of retries and backoff delays. Type: object

Fields of client.retry:

client.retry.retries

Number of retry attempts after a request failure. Type: number

Default value: 3

client.retry.backoffDelay

Initial delay before the first retry attempt. Type: object

Default value: PT0.2S

client.retry.maxBackoffDelay

Maximum delay between consecutive retry attempts. Type: object

Default value: PT3S

client.maxTransactionsPerSecond

The maximum number of transactions allowed per second to control the rate of operations. Type: number

client.maxOutstandingTransactions

The maximum number of outstanding transactions allowed, limiting concurrent operations. Type: number

client.requestTimeout

The timeout duration for client requests to VoltDB, after which a request is considered failed. Type: object

client.authUser

The username for authenticating with the VoltDB cluster. Type: string

client.authPassword

The password for authenticating with the VoltDB cluster. Type: string

client.trustStoreFile

The path to the trust store file used for secure connections to VoltDB. Type: string

client.trustStorePassword

The trust store password. Type: string

exceptionHandler

A custom exception handler to manage errors encountered during data insertion. Type: object

Java dependency management

Add this declaration to your dependency management system to access the configuration DSL for this plugin in Java.

MavenGradle

<dependency>
    <groupId>org.voltdb</groupId>
    <artifactId>volt-stream-plugin-volt-api</artifactId>
    <version>1.4.0</version>
</dependency>

implementation group: 'org.voltdb', name: 'volt-stream-plugin-volt-api', version: '1.4.0'