mlflow-tracking-server | MLFLow Tracking Server based on Docker and AWS S3 | Cloud Storage library

Your mlflow-tracking-server service should have ClusterIP type, not LoadBalancer.

Both pods are inside the same Kubernetes cluster, therefore, there is no reason to use LoadBalancer Service type.

For some parts of your application (for example, frontends) you may want to expose a Service onto an external IP address, that’s outside of your cluster. Kubernetes ServiceTypes allow you to specify what kind of Service you want. The default is ClusterIP.

Type values and their behaviors are:

• ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.

• NodePort: Exposes the Service on each Node’s IP at a static port (the NodePort). A > ClusterIP Service, to which the NodePort Service routes, is automatically created. You’ll > be able to contact the NodePort Service, from outside the cluster, by requesting :.

• LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.
• ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up.

kubernetes.io

The issue here is related to Persistent Volume Claim that is not provisioned by Your minikube cluster.

You will need to make a decision to switch to platform managed kubernetes service or to stick with minikube and manually satisfy the Persistent Volume Claim or with alternative solutions.

The simplest option would be to use helm charts for mflow installation like this or this.

The first helm chart has listed requirements:

Prerequisites

• Kubernetes cluster 1.10+
• Helm 2.8.0+
• PV provisioner support in the underlying infrastructure.

Just like in the guide You followed this one requires PV provisioner support.

So by switching to EKS You most likely will have easier time deploying mflow with artifact storing with s3.

If You wish to stay on minikube, You will need to modify the helm chart values or the yaml files from the guide You linked to be compatible with You manual configuration of PV. It might also need permissions configuration for s3.

The second helm chart has the following limitation/feature:

Known limitations of this Chart

I've created this Chart to use it in a production-ready environment in my company. We are using MLFlow with a Postgres backend store.

Therefore, the following capabilities have been left out of the Chart:

• Using persistent volumes as a backend store.
• Using other database engines like MySQL or SQLServer.

You can try to install it on minikube. This setup would result in artifacts being stored on remote a database. It would still need tweaking in order to connect to s3.

Anyway minikube still is a lightweight distribution of kubernetes targeted mainly for learning, so You will eventually reach another limitation if You stick to it for too long.

Hope it helps.