Skip to content

Mlflow-model

This mlflow-model resource integrates with the MLflow model registry to track and download a machine learning model. It monitors the registry for a new model version and notifies registered listeners, allowing other plugins to automatically reload that model when updates are available.

See the official MLflow documentation for more information.

Model Registry

Connects to an MLflow tracking server and monitors a specific registered model as configured. When a new version is detected, the model is downloaded and made available to other pipeline components.

Model Download

Models are downloaded to a local cache directory and can be reused across pipeline restarts. The cache can be configured with size limits and expiration policies.

.configureResource(MLflowModelResourceConfigBuilder.builder()
    .withTrackingUri(value)
    .withModelName(value)
    .withPollInterval(value)
    .withStage(value)
    .withAuthToken(value)
    .withCacheDirectory(value)
)

resource:
  mlflow-model:
    trackingUri: value
    modelName: value
    pollInterval: value
    stage: value
    authToken: value
    cacheDirectory: value

Java dependency management

Add this declaration to your dependency management system to access the configuration DSL for this plugin in Java.

<dependency>
    <groupId>org.voltdb</groupId>
    <artifactId>volt-stream-plugin-mlflow-api</artifactId>
    <version>1.7.0</version>
</dependency>

implementation group: 'org.voltdb', name: 'volt-stream-plugin-mlflow-api', version: '1.7.0'

Properties

trackingUri

URI to the MLflow tracking server. Required.

Type: string

modelName

Name of the registered model to monitor. Required.

Type: string

pollInterval

Interval for polling the MLflow registry for new model versions. Type: object

Default value: 60s

stage

Specific model stage to monitor (e.g., 'Production', 'Staging'). If not specified, monitors the latest version. Type: string

authToken

Authentication token for MLflow tracking server. Type: string

cacheDirectory

Directory where downloaded model files will be cached. If not specified, a temporary directory will be created. Type: string

JSON Schema

You can validate or explore the configuration using its JSON Schema.

Usage Examples

version: 1
name: MLflowModelPipeline

resources:
  - name: "mlflow-my-classifier"
    mlflow-model:
       trackingUri: "http://localhost:5000"
       modelName: "my-classifier"
       pollInterval: 60s
       cacheDirectory: "/var/models/"

source:
   stdin: {}

processors:
  - onnx:
      modelResource: "mlflow-my-classifier"

sink:
  stdout: {}