15.3. Identifying Export Streams in the Schema

Documentation

VoltDB Home » Documentation » Using VoltDB

15.3. Identifying Export Streams in the Schema

Once you decide what data to export, you are ready to declare your export streams in the schema. Streams are defined in the database schema much like tables, except you use the CREATE STREAM statement instead of CREATE TABLE. So in the case of the flight application, we need to add the export streams to our schema. The following example illustrates (in bold) the addition of a stream for reservations with a subset of columns from the normal reservation table.

    . . .

CREATE TABLE Reservation (
   ReserveID INTEGER UNIQUE NOT NULL,
   FlightID INTEGER NOT NULL,
   CustomerID INTEGER NOT NULL,
   Seat VARCHAR(5) DEFAULT NULL,
   Confirmed TINYINT DEFAULT '0',
   PRIMARY KEY(ReserveID)
);
CREATE STREAM Reservation_final
  EXPORT TO TARGET archive (
   ReserveID INTEGER NOT NULL,
   FlightID INTEGER NOT NULL,
   CustomerID INTEGER NOT NULL,
   Seat VARCHAR(5) DEFAULT NULL
);
     . . .

When you declare your streams in the schema, you need to assign them to the appropriate target, using the EXPORT TO TARGET clause. In our example, the three streams are being exported to the same target, so would name the same target in the declaration, as shown in the following abbreviated example:

CREATE STREAM export_customer 
   EXPORT TO TARGET archive (
        . . .
);
CREATE STREAM export_flight
   EXPORT TO TARGET archive (
        . . .
);
CREATE STREAM reservation_final
   EXPORT TO TARGET archive (
        . . .
);

If a stream does not specify an export target, it is not exported. In the preceding example, export_customer, export_flight, and reservation_final streams are identified as the streams that will be sent to the export target called archive. Note that, even if an export target is specified in the CREATE STREAM statement, inserting data into these streams will have no effect until export is enabled for the archive target in the configuration file.

If you want to export to different locations, you can assign the streams to different targets, then export each stream separately. For example, if you want to export the reservations to a log file but the customer and flight records to an archival database, you can assign the streams to two different targets:

CREATE STREAM export_customer 
   EXPORT TO TARGET archive (
        . . .
);
CREATE STREAM export_flight
   EXPORT TO TARGET archive (
        . . .
);
CREATE STREAM reservation_final
   EXPORT TO TARGET log (
        . . .
);

Note that no changes are required to the client application. The configuration of streams and export targets is all done through the schema and configuration file.

You can also specify whether the streams are partitioned or not using the PARTITION ON COLUMN clause in the CREATE STREAM statement. For example, if an export stream is a copy of a normal data table, it can be partitioned on the same column. However, partitioning is not necessary for export streams. Whether they are partitioned or "replicated", since no storage is associated with the stream, you can INSERT into the stream in either a single-partitioned or multi-partitioned stored procedure. In either case, the export connector ensures that at least one copy of the tuple is written to the export target.