acs-engine | WE HAVE MOVED : Please join us at Azure/aks-engine | Azure library

I am working with Kubernetes on Azure. I created and configured the Azure Kubernetes Service and Azure Container Registry with the Azure command line interface (CLI) - I have a repository full of Azure CLI bash scripts that create/destroy my cloud infrastructure.

I plan to deploy to Kubernetes using Helm. I plan to use Helm Secrets to encrypt secrets and I am hoping I can use Azure Key Vault to store the encryption keys that encrypt/decrypt the "secrets".

I am now trying to create and configure the Azure Key Vault to work with Azure Kubernetes Service. I can see how I could create a Key Vault with the Azure CLI but the documentation for kubernetes-kms suggests that if you configure Kubernetes a certain way then the Key Vault will be automatically created and this will be configured to communicate with Azure Kubernetes Service.

"We have added this feature to aks-engine so that you do not have to worry about any of the manual steps to set this up."

The documentation talks about editing a "kubernetesConfig", I've not seen a "kubernetesConfig" anywhere yet - I guess that is what you would have if you created your components using Azure Resource Manager (ARM) Templates.

How do I get Azure Key Vault working with kubernetes-kms using Azure CLI?

e.g. adding enableEncryptionWithExternalKms to the Kubernetes configuration and adding an objectId attribute to the service principal.

I am experiencing a very complicated issue with Kubernetes in my production environments losing all their Agent Nodes, they change from Ready to NotReady, all the pods change from Running to NodeLost state. I have discovered that Kubernetes is making intensive usage of disks:

My cluster is deployed using acs-engine 0.17.0 (and I tested previous versions too and the same happened).

On the other hand, we decided to deploy the Standard_DS2_VX VM series which contains Premium disks and we incresed the IOPS to 2000 (It was previously under 500 IOPS) and same thing happened. I am going to try with a higher number now.

Any help on this will be appreaciated.

I am attempting to set up a cluster in Azure using acs-engine to build the Kubernetes cluster utilizing VMSS for the agent pools. After the cluster is up I add the cluster-autoscaler to manage 2 dedicated agent pools, 1 cpu and 1 gpu. Scale-down and scale-up work as long as the scale set still has running VMs in them. Both scale sets are set to scale down to 0. With ACS I have set these 2 scale sets up with taints and custom labels. Once the scale set has scaled down to 0, I am unable to get the autoscaler to spin back up a node when a new pod is scheduled. I am not sure what I'm doing wrong or if I am missing some config, label, taint, etc. I just started using kubernetes recently.

Below is my acs-engine json, pod definition and the logs from the autoscaler and pod describe.

Output from kubectl logs -n kube-system cluster-autoscaler-5967b96496-jnvjr

I created Kubernetes cluster using ACS-engine in Azure and installed Ingress controller.

I deployed a service:

• Inside the cluster running the command curl :/myservice give the expected response.

I created a VM with NGinx in order to get external access and using the external IP of the VM such external access to the service exists.

My Goal:

• Replace the VM with Azure IAAS Loadbalancer.

Steps that i did using the UI:

1. Click Create resources -> Create public IP address -> set name of the IP -> set assignment Static -> define the resource group.

2. Click create resources -> Networking -> Load Balancer -> select the publicip that created in step #1 -> Use the same resource group.

3. Created Health probe -> protocol HTTP, port 32597 (Ingress port) and path /myservice

4. Defined the backend pool to the Availabilityset of the Kubernetes nodes (agents).

5. Created Inbound NAT rule: Service: Custom. Port: 32597. Availabilityset: Kubernetes nodes (agents)

Unlike the VM i cannot reach the website and i do not see in the Loadbalncer logs any information.

Questions:

1. How to get the traffic logs ?
2. Can i run TCPDUMP ?
3. How to fix the issue ?

Thank you.

I created a Docker image based on microsoft/dotnet-framework of a C#.NET console application built for Windows containers, then ensured I can run the image in a container locally. I successfully pushed the image to our Azure Container registry. Now I'm trying to create a deployment in our Azure Kubernetes service, but I'm getting an error:

Failed to pull image "container-registry/image:tag": rpc error: code = Unknown desc = unknown blob

I see this error on my deployment, pods, and replica sets in the Kubernetes dashboard.

We already have a secret that works with the azure-vote app, so I wouldn't think this is related to secrets, but I could be wrong.

So far, I've tried to create this deployment by pasting the following YAML into the Kubernetes dashboard Create dialog:

With acs-engine I have created a k8s cluster with a custom vnet. The cluster was deployed and the pods are running. When I do a kubectl get nodes or get pod I get a reply. But when I use exec to get into a pod or use helm install then I get the error:

Error from server: error dialing backend: dial tcp: lookup k8s-agentpool on 10.40.1.133:53: server misbehaving

I used the following json file to create the arm templates: acs-engine.json

When not using a custom vnet then the default azure dns is used and with a custom vnet our own dns servers are used. Is the only option to register all masters and agents to the dns server?

I am using AKS and when I create a service of type LoadBalancer, it creates a service and allocates and IP but the following requests to create services of type loadbalancer the IP does not get allocated and it shows state forever

I verified that the quotas for public IP are witin range. Is this a limitation that I am hitting? How should I go abot debugging this?

This is a relevant link https://github.com/Azure/acs-engine/issues/737

I see this when I descibe the service

I have deployed a Kubernetes cluster to a custom virtual network on Azure using acs-engine. There is an ASP.NET Core 2.0 Kestrel app running on the agent VMs and the app is accessed over VPN through a Service of the Azure internal load balancer type. Now I would like to enable HTTPS on the service. I have already obtained a domain name and a certificate but have no idea how to proceed. Apparently configuring Kestrel to use HTTPS and copying the certificate to each container is not the way to go.

I have checked out tutorials such as ingress on k8s using acs and configure Nginx Ingress Controller for TLS termination on k8s on Azure but both of them end up exposing a public external IP and I want to keep the IP internal and not accessible from the internet. Is this possible? Can it be done without ingresses and their controllers?

I have 2 clusters running in Azure for 2 different Availability Zones and I would like to cluster the etcd masters following https://kubernetes.io/docs/admin/high-availability/#replicated-api-servers .

I created the discovery token for 3 masters. When I try to init etcd container it fails: