admission-control | helpful micro-framework for writing Kubernetes | Continuous Deployment library

I was able to reproduce your scenario but using Helm.

Original Replication Steps to create 2 Nginx Ingress in one Cluster

Create two namespaces for development: dev1, dev2 and two for ingress: ing1, ing2.

It's kind of related to how the user is authenticated to the cluster and how they get a kubeconfig file.You can put a group in the client certificate or the bearer token that kubectl uses from the kubeconfig. Ahead of time you can define a clusterrole having a clusterrolebinding to that group which gives them permission to certain verbs on certain resources(for example ability to create namespace)

Additionally you can use an admission webhook to validate if the user is supposed to be part of that group or not.

Replace the forward slash (/) in kubernetes.io/ingress.class with ~1.

Your command should look like this,

First of all:

$ kubectl run --name=nginx hello-world

You did not specify image name of the pod. Correct syntax should be:

$ kubectl run --image=nginx NAME_OF_DEPLOYMENT

As said above commands will try to create a deployment.

The issue you are encountering is most probably connected with:

• Not working/turned on admission controller

On newly created Kubernetes cluster with pod security policy turned on you should not be able to spawn any pod regardless of your privileges.

Pod security policy control is implemented as an optional (but recommended) admission controller. PodSecurityPolicies are enforced by enabling the admission controller, but doing so without authorizing any policies will prevent any pods from being created in the cluster.

-- Kubernetes.io: enabling pod security policies

Admission controller as well as pod security policy and RBAC are strongly connected with solutions you are working with. You should refer to documentation specific to your case.

For example:

• Newly created GKE cluster with pod security enabled and none PSP configured will not create pods. It will display a message: Unable to validate against any pod security policy: []

Warning: If you enable the PodSecurityPolicy controller without first defining and authorizing any actual policies, no users, controllers, or service accounts can create or update Pods. If you are working with an existing cluster, you should define and authorize policies before enabling the controller.

-- GKE: Pod security policies and how to enable/disable it

• Newly created Kubernetes cluster with kubespray (with pod security policy variable set to true when provisioning and running on Ubuntu) will have a restrictive PSP created and it will have a MustRunAsNonRoot parameter inside the PSP.

There is another issue with NGINX pod. NGINX image will try to run as root user inside of the pod. Admission controller with PSP configured with:

As a workaround you can use standard GNU/Linux diff command in the following way:

This problem is due to kubernates tls certificates expiry. Rancher version v2.0.8 does not have auto refresh mechanism for ssl/tls certificates. I have upgraded to v2.2.8, and the issue is fixed now. In v2.2.8 they have provided a solution for refreshing of kubernates certificates from the console.

I see you are trying to understand what is the ordering of execution of mutating webhooks.

I have found this piece of code in kubernetes repo. Based on this you can see that these are sorted by name of a webhook to have a deterministic order.

A single ordering of mutating admissions plugins (including webhooks) does not work for all cases, so take a look at mutating plugin ordering section in Admission webhook proposal for explanation how its handled.

Also notice there are no "pod only endpoints" or "endpoints that get called for pods". Let's say you have your webhook server and want to mutate pods and your server has only one endpoint: /. If you want to mutate pods with it you need to specify it under rules. So setting rules[].resources: ["pods"] and rules[].operations: ["CREATE"] in your webhook config will run your mutating webhook whenever there is pod to be created.

Let me know it it helped.

This may caused by your certificate file generate encount warning,you should use new version of cfssl(above v1.2),and make sure have no warning.This is cause by this tip when using cfssl(v1.3) to generate certificate:

The token format in your bootstrap.kubeconfig.yaml looks different than usual tokens that are generated by kubeadm.

According to the article Authenticating with Bootstrap Tokens:

Token Format

Bootstrap Tokens take the form of abcdef.0123456789abcdef. More formally, they must match the regular expression [a-z0-9]{6}.[a-z0-9]{16}.

The first part of the token is the “Token ID” and is considered public information. It is used when referring to a token without leaking the secret part used for authentication. The second part is the “Token Secret” and should only be shared with trusted parties.

Consider reading the previous and this article to understand how the Bootstrap Token idea is implemented.