How did I setup my first Kubernetes cluster…

March 10, 2018Damian Czaja

Don’t know how about you, but I love Docker. I makes deploying applications a lot easier. Wanna setup a game server? Need a database? Docker allows to spin it up in matter of seconds.

I have a VPS at OVH, which I use as a testing environment and to run a few services for me and my friends. There’s a Teamspeak Server and sometimes game servers for multiplayer games like Minecraft. It you need a TS server to play with your friends, you’re welcome at ts.myhightech.org. 🙂

As I’m a big fan of DevOps/automation, I wanted to have something like Marathon there to easily deploy/upgrade services without having to write Ansible scripts or SSH manually to the server. I was running recently a workshop about authorization at work and prepared a few demo apps, which I run on my VPS. Ansible did the job, but I wanted better. Everyone’s talking now about Kubernetes, so I had to try it out.

My target was: setup a working Kubernetes cluster and deploy a Teamspeak Server there. I will also run there a website, which shows, who’s online on Teamspeak.

For those of you who don’t know Teamspeak – it’s a VoIP application used for ex. by gamers. It stores some information about channels and users in a database. In my setup it’s SQLite, so basically a file. So besides deploying Teamspeak, I had to make the database file somehow persistent between containers, in case of a potential crash.

Kubernetes setup

I spinned up three VPS from Digital Ocean (one for master with 1 CPU and 1GB RAM and two for nodes with 1 CPU and 2GB RAM) with CentOS 7. To provision there a great tools called kubeadm, which does most of the job for you. You can follow the guide here (pro tip: watch for the supported Docker versions, cgroup driver and disable swap!).

But there’s even a better solution… kubeadm-ansible! It’s a clone-and-play solution to provision and create an Kubernetes cluster and worked great in my case. The only thing you have to do is to put your hosts in the inventory file and that’s it! It will install Docker, Kubernetes, connect nodes to the cluster and setup a pod network (Calico by default, but you can change it to Flannel in group_vars/all.yml).

After that, output of kubectl get nodes from my master node was:

Run your first app

OK, so let’s try the new cluster out and launch a dummy HTTP server. For this I needed three things:

  • Kube config file from ~/.kube/config on the master node
  • kubectl CLI tool
  • a deployment YAML definition for the HTTP server

For the first two bullets you can use the master node, which already has the config file and kubectl, but I found it easier to install kubectl on my PC and download the kube config file from the master node and place it in ~/.kube/config. After that output from kubectl get nodes on you PC should be the same like on the master node.

I decided to use NGINX and prepared the following YAML file:

It basically defines a deployment and exposes the HTTP port of NGINX on the public IP. I run it with kubectl create -f static_web.yml and after a few moments NGINX was running on http://165.227.149.65.

GlusterFS

I was now able to run deployments, but I had no solution to make the data inside the container persistent, so the Teamspeak database won’t be lost, when the container crashes. I could simply mount it to the local storage on the node, but it won’t work, if the Teamspeak pod starts on the other node. That’s where GlusterFS comes is. GlusterFS is a distributed file system. It allows to setup a Gluster cluster, which synchronizes the data between the nodes and allows to mount the virtual volumes on other machines.

I ordered 3 x 10GB volumes, mounted each to one of the three VPSes and installed GlusterFS on each by following the installation guide.

After that I connected to nodes to a cluster

At this point I had an Gluster volume named teamspeak with 3 bricks, which is ready to be mounted to a deployment. To allow Kubernetes connect to the cluster you needed to register an Endpoint resource in Kubernetes:

and run kubectl apply -f prepare.yml

Deploy Teamspeak

I have prepared a Docker image for Teamspeak. It’s built by Gitlab CI and is available under registry.gitlab.com/teamspeak/teamspeak:latest. To deploy it I prepared the following deployment definition:

It defines the Teamspeak container, mounts the Gluster teamspeak volume and exposes 30033/TCP and 9987/UDP ports on the public IP 165.227.153.80.

I run kubectl apply -f teamspeak.yml and after a few moments the server was up and running:

Well, in my case it’s not an fresh Teamspeak server, cause I uploaded backups from my previous server to the Gluster volume… which proves GlusterFS is working!

Summary

By using kubeadm-ansible and GlusterFS I was able to setup a working Kubernetes cluster in one evening! I consider it as an great playground for any kind of development or research work. It requires small resources and allows to deploy and manage applications in a simple way (I’ve never had to SSH to the nodes after Kubernetes and GlusterFS were setup).

One last thing I have to share: for application development there is a great tool called Telepresence. It allows you to connect your PC to the Kubernetes cluster, like it were a pod running on the cluster. Simply said: you can connect from your local PC to services running on the cluster and pods on the cluster can connect services running on your PC via Kubernetes DNS. It makes local development extremely easier and that’s a tool I miss, when working on a Mesos cluster.

1 Shares

Comments (1)

Leave a comment

Your email address will not be published. Required fields are marked *

Prev Post