multi-node | node provides launching of multiple NodeJS processes | Runtime Evironment library

In the latest version (ES8), security is on by default (i.e. SSL/TLS).

If you're accessing from the browser, just use https instead of http:

You shouldn't consider this a production environment. The guides are examples, often for lab environments, and testing scenarios with the application. I would not consider them production ready, and compose is often not considered a production grade tool since everything it does is to a single docker node, where in production you typically want multiple nodes spread across multiple availability zones.

Unfortunately, there is no other way. You have to use -N.

Even if you use -n 1 (instead of 4) there will be a warning:

No, it is not the expected behavior for Kubernetes. Pods should be accessible for all the nodes inside the same cluster through their internal IPs. ClusterIP service exposes the service on a cluster-internal IP and making it reachable from within the cluster - it is basically set by default for all the service types, as stated in Kubernetes documentation.

Services are not node-specific and they can point to a pod regardless of where it runs in the cluster at any given moment in time. Also make sure that you are using the cluster-internal port: while trying to reach the services. If you still can connect to the pod only from node where it is running, you might need to check if something is wrong with your networking - e.g, check if UDP ports are blocked.

EDIT: Concerning network policies - by default, a pod is non-isolated either for egress or ingress, i.e. if no NetworkPolicy resource is defined for the pod in Kubernetes, all traffic is allowed to/from this pod - so-called default-allow behavior. Basically, without network policies all pods are allowed to communicate with all other pods/services in the same cluster, as described above. If one or more NetworkPolicy is applied to a particular pod, it will reject all traffic that is not explicitly allowed by that policies (meaning, NetworkPolicythat both selects the pod and has "Ingress"/"Egress" in its policyTypes) - default-deny behavior.

What is more:

Network policies do not conflict; they are additive. If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod is the union of what the applicable policies allow.

So yes, it is expected behavior for Kubernetes NetworkPolicy - when a pod is isolated for ingress/egress, the only allowed connections into/from the pod are those from the pod's node and those allowed by the connection list of NetworkPolicy defined. To be compatible with it, Calico network policy follows the same behavior for Kubernetes pods. NetworkPolicy is applied to pods within a particular namespace - either the same or different with the help of the selectors.

As for node specific policies - nodes can't be targeted by their Kubernetes identities, instead CIDR notation should be used in form of ipBlock in pod/service NetworkPolicy - particular IP ranges are selected to allow as ingress sources or egress destinations for pod/service.

Whitelisting Calico IP addresses for each node might seem to be a valid option in this case, please have a look at the similar issue described here.

AWS Batch would fit your requirements better.

Airflow is a workflow orchestration tool, it's used to host many jobs that have multiple tasks each, with each task being light on processing. Its most common use is for ETL, but in your use case you would have an entire Airflow ecosystem for just a single job, which (unless you manually broke it out to smaller tasks) would not run multi-threaded.

AWS Batch on the other hand is for batch processing, and you can more finely-tune the servers/nodes that you want your code to execute on. I think in your use case it would also work out cheaper than Airflow too.

We can recursively search all nodes until we find the match:

I've got the same error as you when I tried to run k0s token create --role=worker on the worker node.

You need to run this command on the master node:

Next, you need to create a join token that the worker node will use to join the cluster. This token is generated from the control node.

First you need to run k0s token create --role=worker on the master node to get a token and later use this token on the worker node:

On the worker node, issue the command below.

The secret key must be the same as the distribution namespace. If you want to connect your docker secret to kubernetes you can use below method.

Create a Secret based on existing Docker credentials (link)

I ended up using kubeadm and set up a master and worker node using 2 VMs. This allowed me to specify the versions I want on the worker node. Building a base image on kind should also work as user Mikolaj.S mentioned