MinIO Kubernetes Operator

Kubernetes Version 1.21.0

Starting with v5.0.0, MinIO requires Kubernetes 1.21.0 or later for both the infrastructure and the kubectl CLI tool.

For Kubernetes 1.25.0 and later, MinIO supports deploying in environments with the Pod Security admission (PSA) restricted policy enabled.

Kustomize and kubectl

Kustomize is a YAML-based templating tool that allows you to define Kubernetes resources in a declarative and repeatable fashion. Kustomize is included with the kubectl command line tool.

This procedure assumes that your local host machine has both the matching version of kubectl for your Kubernetes cluster and the necessary access to that cluster to create new resources.

The default MinIO Operator Kustomize template provides a starting point for customizing configurations for your local environment. You can modify the default Kustomization file or apply your own patches to customize the Operator deployment for your Kubernetes cluster.

Kubernetes TLS Certificate API

For any of these circumstances, the MinIO Operator requires that the Kubernetes kube-controller-manager configuration include the following configuration settings:

• --cluster-signing-key-file - Specify the PEM-encoded RSA or ECDSA private key used to sign cluster-scoped certificates.

• --cluster-signing-cert-file - Specify the PEM-encoded x.509 Certificate Authority certificate used to issue cluster-scoped certificates.

The Kubernetes TLS API uses the CA signature algorithm for generating new TLS certificate. MinIO recommends ECDSA (e.g. NIST P-256 curve) or EdDSA (e.g. Curve25519) TLS private keys/certificates due to their lower computation requirements compared to RSA. See Supported TLS Cipher Suites for a complete list of supported TLS Cipher Suites.

If the Kubernetes cluster is not configured to respond to a generated CSR, the Operator cannot complete initialization. Some Kubernetes providers do not specify these configuration values by default.

To check whether the kube-controller-manager specifies the cluster signing key and certificate files, use the following command:

kubectl get pod kube-controller-manager-$CLUSTERNAME-control-plane \
  -n kube-system -o yaml

• Replace $CLUSTERNAME with the name of the Kubernetes cluster.

Confirm that the output contains the highlighted lines. The output of the example command above may differ from the output in your terminal:

 spec:
 containers:
 - command:
     - kube-controller-manager
     - --allocate-node-cidrs=true
     - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
     - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
     - --bind-address=127.0.0.1
     - --client-ca-file=/etc/kubernetes/pki/ca.crt
     - --cluster-cidr=10.244.0.0/16
     - --cluster-name=my-cluster-name
     - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
     - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key
 ...

Operator Namespace

ToDO

Tenant Namespace

When you use the Operator to create a tenant, the tenant must have its own namespace. Within that namespace, the Operator generates the pods required by the tenant configuration.

Each Tenant pod runs three containers:

• MinIO Container that runs all of the standard MinIO functions, equivalent to basic MinIO installation on baremetal. This container stores and retrieves objects in the provided mount points (persistent volumes).

• InitContainer that only exists during the launch of the pod to manage configuration secrets during startup. Once startup completes, this container terminates.

• SideCar container that monitors configuration secrets for the tenant and updates them as they change. This container also monitors for root credentials and creates an error if it does not find root credentials.

Starting with v5.0.6, the MinIO Operator supports custom init containers for additional pod initialization that may be required for your environment.

The tenant utilizes Persistent Volume Claims to talk to the Persistent Volumes that store the objects.