9.4. Modifying Both the Procedure and Interval

Documentation

VoltDB Home » Documentation » Guide to Performance and Customization

9.4. Modifying Both the Procedure and Interval

To allow full customization of the task — including the procedure, its arguments, and the interval between invocations — you create a custom task that implements the ActionScheduler interface. The custom task consists of the Java class you create, and the SQL statements to load and declare the task. At run time, for each invocation of the task, VoltDB uses the custom class to determine what procedure to invoke, what arguments to pass to the procedure, and how long to wait before invoking it.

Let's look at an example. The previous examples create tasks for purging old sessions from a database. But how can you tell that these tasks work without a real application? One way is to create a test database with tasks to emulate the expected workload, generating new sessions on an ongoing basis.

For our example, we want a task that generates sessions, but not so many the number of sessions created exceeds the number deleted. So our custom task needs to perform two actions:

  1. Check to see how many session records are in the database.

  2. If the task hasn't reached its target goal, generate up to 1,000 new sessions. Then repeat step 1.

In other words, we need two separate procedures: one to count the number of session records and another to insert a new session record. To count all session records in the database, the first procedure, CountSessions, must be multi-partitioned. (It also checks for the maximum value of the session ID so it can generate an incrementally unique ID.) But the second procedure, AddSession, can be partitioned since it is inserting into a partitioned table. This way the task reproduces the actions and performance of a running application.

CREATE PROCEDURE CountSessions AS
    SELECT COUNT(*), MAX(sessionID) FROM session;

CREATE PROCEDURE AddSession PARTITION ON TABLE SESSION COLUMN sessionID AS
    INSERT INTO session (sessionID,userID) VALUES(?,?);

The custom task will decide which procedure to invoke, and how long to wait between invocations, based on the results of the previous execution. There are many ways to do this, but for the sake of example, our custom task uses two separate callback methods: one to evaluate the results of the CountSessions procedure and one to evaluate the results of the AddSession procedure, as described in the next section.

9.4.1. Designing a Java Class That Implements ActionScheduler

The majority of the work of a custom class is performed by a Java class that implements a task interface; in this case, the ActionScheduler interface. It must be a static class whose constructor takes no arguments. The class must, at a minimum, declare or override three methods:

1

initialize()

The initialize() method is called first and receives a helper object inserted automatically by the subsystem that manages tasks, plus any parameters defined by the task definition in SQL. In our example there is one parameter in the task definition: the target number of records to create. So the initialize() method has a total of two arguments.

2

GetFirstInterval()

The getFirstInterval() method is called when the task starts; that is, when the task is first defined or when the database starts or resumes. The method must return a ScheduledAction object, which specifies the length of time to wait until the first execution of the task, a callback to invoke after the task runs, plus the name of the procedure and any arguments the procedure requires. In our example, we initialize the ScheduledAction object with no wait time (that is, an interval of zero), the callback for the CheckSessions procedure, and the procedure itself.

3

callback methods

After each iteration of the task, VoltDB invokes the specified callback procedure. In our example, there are two callback methods:

  • checkcallback — The task specifies this as the callback method to invoke after each iteration of the CheckSessions stored procedure. The callback checks the results for the number of session records. (it also saves the highest value of the session ID so it can generate an incrementally unique ID for each new record.) If the number of records is less than the goal, it schedules the AddSession procedure as the next task, specifying loadcallback() as the callback method. If the goal has been met, no more records need to be added so the callback backs off on the frequency (increasing the interval) and schedules the CheckSessions procedure again.

  • loadcallback — After each execution of the AddSession procedure, the callback reduces the batch size by one. If the batch is not complete, the callback reschedules the AddSession procedure, waiting 2 milliseconds. If the batch of inserts has been completed (that is, the batch size is down to zero), the callback schedules the CheckSessions procedure, specifying its callback method checkcallback() to see if there is now a full complement of session records.

Two things to note about this process are that if the session table is filled to the specified goal, the checkcallback uses the ability to customize the interval to reduce the frequency of checking — to minimize the impact on other transactions. Also, besides scheduling different stored procedures at different times, it passes different arguments as well, inserting a unique session ID and randomized user ID for each AddSession invocation.

Example 9.3, “Custom Task Implementing ActionScheduler” shows the completed example task class, with the key elements highlighted.

Example 9.3. Custom Task Implementing ActionScheduler

package mytasks;

import java.util.Random;

import java.util.concurrent.TimeUnit;
import org.voltdb.VoltTable;
import org.voltdb.client.ClientResponse;
import org.voltdb.task.*;

public  class LoadSessions implements ActionScheduler {

  private TaskHelper helper;
  private long batch, wait, nextid, goal;

  public void initialize(TaskHelper helper, long goal) {              1
    this.goal = goal;
    this.wait = 0;
  }

  public ScheduledAction getFirstScheduledAction() {                  2
    return ScheduledAction.procedureCall(0, TimeUnit.MILLISECONDS, 
       this::checkcallback, "CountSessions");
  }

  /*
   * Callbacks to handle the results of the check task 
   * and the load task
   */

  private ScheduledAction checkcallback(ActionResult result) {        3
  
    ClientResponse response = result.getResponse();
    VoltTable[] results = response.getResults();
    long recordcount = results[0].fetchRow(0).getLong(0);
    this.nextid = results[0].fetchRow(0).getLong(1);
    
    if (recordcount == 0) this.nextid = 0; /* start fresh*/
    this.batch = this.goal - recordcount;

     if (this.batch > 0) {
       /* Start loading data. Max batch size is 1,000 records */
       this.batch = (this.batch < 1000) ? this.batch : 1000;
       this.nextid++;
       return ScheduledAction.procedureCall(2, TimeUnit.MILLISECONDS, 
               this::loadcallback, "AddSession", 
               this.nextid,randomuser());
    } else {
       /* schedule the next check. */
       this.wait += 500;
       if (this.wait > 60000) this.wait = 60000;
       return ScheduledAction.procedureCall(this.wait, TimeUnit.MILLISECONDS, 
              this::checkcallback, "CountSessions");
    }
  }

  private ScheduledAction loadcallback(ActionResult result) {        3
    this.batch--;
    if (this.batch > 0 ) {
       /* Load next session */
       this.nextid++;
       return ScheduledAction.procedureCall(2, TimeUnit.MILLISECONDS, 
               this::loadcallback, "AddSession", 
               this.nextid,randomuser());
    } else {
       /* schedule the next check. */
       System.out.println(" Done.");
       this.wait = 0;
       return ScheduledAction.procedureCall(this.wait, TimeUnit.MILLISECONDS, 
            this::checkcallback, "CountSessions");
    }
  }
  
  private long randomuser() { return new Random().nextInt(1001); }

}

9.4.2. Compiling and Loading the Class into VoltDB

Once you complete your Java source code, you compile, debug, and package it into a JAR file the same way you compile and package stored procedures. You can package tasks, procedures, and other classes (such as user-defined functions) into a single or separate JARs depending on your application and operational needs. The following example compiles the Java classes in the src folder and packages it into the JAR file sessiontasks.jar:

$ javac -classpath "/opt/voltdb/voltdb/*" \
        -d ./obj  src/*.java
$ jar  cvf  sessiontasks.jar -C obj . 

You then load the classes from the JAR file into VoltDB using the sqlcmd LOAD CLASSES directive:

LOAD CLASSES sessiontasks.jar;

9.4.3. Declaring the Task

Once the custom class is loaded into the database, you can declare the task and start it running. You declare the task using the CREATE TASK statement, replacing both the ON SCHEDULE and PROCEDURE clauses with a single FROM CLASS clause specifying the classpath of your new class. In our example, the custom task also requires one argument: the target value for the maximum number of session records to create. The following statement creates the custom task with a goal of 10,000 records.

CREATE TASK loadsessions 
  FROM CLASS mytasks.LoadSessions WITH (10000)
  RUN ON DATABASE;

Note that the task is defined to RUN ON DATABASE. This means only one instance of the task is running at one time. However, the stored procedures will run as defined; that is, the CheckSessions procedure will run as a multi-partitioned procedure and each instance of AddSession will be executed on a specific partition based on the unique partition ID passed in as an argument at run-time.

The task starts as soon as it is declared, unless you include the DISABLE clause. Alternately, you can use the ALTER TASK statement to change the state of the task. For example, the following statement disables our newly created task:

3> ALTER TASK loadsessions DISABLE;