Skip to content

S3

The s3 resource configures and exposes the Amazon S3 client instance.

Advanced users that need direct access to Amazon S3 do not need to configure and instantiate it manually. VoltSP uses built-in mechanisms to provide a production quality setup.

User can relay on VoltSP standard configuration options to customize this resource.

This resource provides access to Amazon S3 operations through the AWS SDK for Java.

.configureResource(S3ResourceConfigBuilder.builder()
    .withRegion(value)
    .withEndpoint(value)
    .withCredentials(builder -> builder
        .withAccessKey(value)
        .withSecretKey(value)
        .withProfileName(value)
        .withFromContainer(value)
        .withFromInstanceProfile(value)
        .withRoleArn(value)
        .withRoleSessionName(value)
    )
)

resource:
  s3:
    region: value
    endpoint: value
    credentials:
      accessKey: value
      secretKey: value
      profileName: value
      fromContainer: value
      fromInstanceProfile: value
      roleArn: value
      roleSessionName: value

Java dependency management

Add this declaration to your dependency management system to access the configuration DSL for this plugin in Java.

<dependency>
    <groupId>org.voltdb</groupId>
    <artifactId>volt-stream-plugin-aws-api</artifactId>
    <version>1.0-20250910-124207-release-1.5.3</version>
</dependency>

implementation group: 'org.voltdb', name: 'volt-stream-plugin-aws-api', version: '1.0-20250910-124207-release-1.5.3'

Properties

region

The AWS region where the S3 bucket is located (e.g., us-east-1). Type: string

Default value: us-east-1

endpoint

The S3 endpoint URL. This is optional and can be used to specify a custom endpoint for S3 compatible services. Type: string

credentials

Optional credential configuration for s3 client.

VoltSP supports following ways of retrieving authentication credentials: - Using access key and secret key. This is the most basic way of authentication. It is also the least secure way due to the need to store secret key in the VoltSP configuration. - Using a profile from the AWS credentials file. - Using EC2 instance profile credentials. - Using container credentials (for ECS). - Using web identity token credentials (for AWS STS).

If no credentials are configured then VoltSP will use the default credential provider chain as defined by AWS Java SDK. This will look for credentials in the following order: 1. Environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) 2. Java system properties (aws.accessKeyId and aws.secretKey) 3. Web Identity Token credentials from the environment or container 4. Credentials file at the default location (~/.aws/credentials) 5. ECS container credentials 6. EC2 Instance profile credentials

Type: object

Fields of credentials:

credentials.accessKey

The AWS access key ID for authentication when using basic credentials. Type: string

credentials.secretKey

The AWS secret access key for authentication when using basic credentials. Type: string

credentials.profileName

The AWS profile name to use when using profile based authentication. This loads credentials from a profile file. Type: string

credentials.fromContainer

Credentials will be read from the container metadata service Type: boolean

credentials.fromInstanceProfile

Credentials will be read from the instance metadata service Type: boolean

credentials.roleArn

The role ARN to assume when using web identity token based authentication. Type: string

credentials.roleSessionName

The role session name to use when using web identity token based authentication. Type: string

Usage Examples

S3ResourceConfigBuilder.builder()
   .withRegion("us-east-1")
   .withEndpoint("https://s3.amazonaws.com")
   .withCredentials(credentialsBuilder -> credentialsBuilder
       .withAccessKey("your-access-key")
       .withSecretKey("your-secret-key"))
   .build();

resources:
    - name: s3-client
      s3:
        region: "us-east-1"
        endpoint: "https://s3.amazonaws.com"
        credentials:
          access-key: "your-access-key"
          secret-key: "your-secret-key"

source:
    stdin: {}

sink:
    stdout: {}