Kubernetes Cluster

Link: https://gitlab.com/ansible-labs/kubernetes-cluster

When working on my thesis, I had to deploy a Kubernetes cluster in the private infrastructure of INESC-ID. I could SSH into physical machines but had to deploy virtual machines on top of them (using VirtualBox) in order to really use them.

Since I had to deploy a Kubernetes cluster, I had to study the differences between the different node types of K8s and how they communicated with each other.

I first started with multiple Virtual Machines (VMs) in one physical machine, and everything seemed to run pretty smoothly, but when I had to scale multiple machines I suddenly realised I had to come up with a solution that allowed me to have virtual machines communicate with others on a different physical machine. I tried to work with Wireguard but it showed itself impracticable since I needed high availability (more than one Kubernetes master node) and it did not support mesh networking. I also tried Nebula but it did not work as well because it couldn't traverse the NATs that were in place by VirtualBox. The final solution was to use a bridge adapter, and it worked flawlessly.

Since the infrastructure I used in a timeframe couldn't be the same I would use in a different one, I had to be able to ramp up the cluster easily in a machine with completely different specifications. To help in this, I decided I wanted to configure the cluster using a YAML file (config.yml) which can be understood both by Ansible and by Vagrant.

I ended up using a VM only for load balancing because it is the recommended way to ensure high availability, and it showed itself quite handy as sometimes I found VMs in a very weird state: responding to ping, able to initiate an SSH connection but were completely broken; and if it was a master node, it could ruin my workload.

The configuration file allows to set the number of master nodes, worker nodes, define wheter or not to deploy a load balancer as well as hardware specifications for each VM and their image.

This project played a crucial role in my thesis and I hope it can help others to play around with Kubernetes in their computer or using a bigger infrastructure like I did.