By default, services running on Kubernetes pods are only accessible within the Kubernetes cluster on which they run. You can use services such as Consul to create a virtual tunnel between disparate Kubernetes clusters so that services in both can interoperate as if they were within the same network context.

An alternative is to use Kubernetes load balancers to make the VoltDB cluster accessible outside the Kubernetes cluster. This allows XDCR clusters in different network domains create an XDCR connection through external IP addresses. However, to make this happen, the load balancers need to be able to assign reliable and predictable network addresses for each Kubernetes pod. If you have sufficient control of the Kubernetes installation or of the load balancer service in use, the following technique may be of value to you.

This technique does not use the load balancers for their usual role, balancing the load between multiple pods. Instead, the load balancers are solely used to provide externally accessible IP addresses. The idea is that one load balancer is assigned to each VoltCB cluster pod. The load balancer presents a consistent and routable external IP address for the pod, despite potential change in the internal pod address.

To use load balancers, the cluster specification in the Helm chart must specify the service type LoadBalancer and a list of available IP addresses. For example:

cluster:
  serviceSpec:
    dr:
      type: LoadBalancer
      servicePerPod: true
      availableIPs:
      - 34.73.217.41
      - 35.231.99.27
      - 104.196.108.213

During deployment, the VoltDB operator creates a service for each VoltDB pod, with appropriate labelling for the service to select its unique pod. The operator allocates an IP address from the list and assigns it as the load balancer IP address of the service. Service creation and IP allocation is performed for both the initial deployment of the cluster and elastic addition of nodes.

For this operation to work, the load balancer service itself must be available within the Kubernetes environment and must support setting the external addresses. To assist with this operation, the VoltDB charts support passing user-defined annotations to the load balancer service as part of the configuration.

Once the cluster has been configured and started, the other XDCR cluster can be configured using the one or more of the IP addresses in the pool as the source for its connection, For example:

cluster:
  config:
    deployment:
      dr:
        id: 2
        role: xdcr
        connection:
          enabled: true
          source: \
   "34.73.217.41:5555,35.231.99.27:5555,104.196.108.213:5555"