Is Kubernetes on Premise viable? | Towards Data Science

Public cloud providers are not always a choice. As cloud a lover, I prefer to use the cloud any time I can, but there are situations where simply we cannot. It can be due to data protection requirement, cost optimization, or just our boss that fear the changes. We probably would like It wasn’t so, but I’m sorry to say that we cannot always change the world. So, what to do in this case? The solution to reject Kubernetes advantages or avoiding container in favor of VMS is not an option at all. It easy to understand that the only option for us, k8s addicted, is to deploy it in on-premise. In this article, I will explain what are the critical points and what you have to take into account before starting this journey.

Why should I use Kubernetes on-premise?

The honest answer is that you shouldn’t. With cloud prices and quality use, a cloud-based solution is the better solution to reduce cost, maintain high-quality standards, and reduce time to market for new installations. The matter is that, even if Kubernetes on cloud is the best alternative, it is not always applicable.

Many organizations simply cannot use the cloud. It may be due to questionable security policy, regulations or data policies, or the unavailability of some cloud services in their country. Or it can be the CIO that bought a new data center and want to give a sense to this investment, using it all company-wide.

It doesn’t matter the reason. The question is that, simply, in some cases, Kubernetes must be kept on-premise. I think that coming back to the VMs age is not an alternative.

What matters using Kubernetes On-Premise

Kubernetes is a tool. Yes, a big, sophisticated, powerful tool. But always a tool. The basics for running tools are:

People
Know how to use it
Have all the tools requirements

If you need to knock a nail into the wood, you need a hammer (tool), a man who knows what to do (you), and a nail (tools requirements). Kubernetes (tool) is quite the same. You will need a man or a team who know what to do with him (setup, maintenance, security) and an infrastructure to run it (tools requirements).

I hope I haven’t hurt anyone making a comparison between a hammer and Kubernetes My intention wasn’t to diminish the worth of Kubernetes, but just introduce the list of critical things we have to tasks care: people and infrastructure. Let’s jump into the next chapters for a deep dive.

Infrastructure Requirements

Theoretically, you can run Kube on your laptop but is not hard to understand that for scaling you will need something more. In a mixed scenario, when you already have hypervisors for deploying VMs, using them for hosting the k8s server is the simpler way. As it is clear that hardware used matters, you will understand that how you set up the infrastructure is also important. Good hardware with a bad configuration may bring to bad performances.

Redundancy

If you want to have a reliable solution is better to use almost three. Using multiple nodes, you can balance the workload and the internal services, like kube-apiserver, etcd, kube-scheduler, and kube-controller-manager, and so on.

Separate master from containers

Master components can run in the same machine of containers, but it is a best practice to isolate them to avoid resource disputes.

Minimum requirements

Kubernetes is designed to be recoverable from failures without losing any data. This characteristic depends on the hardware resource availability. The best practices identify this minimum set of resources as three nodes with 2GB RAM and 8GB SSD. As you can imagine, hardware performance will impact the Kubernetes performance. An SSD or equivalent performance is recommended. Of course, in a production environment, it is a good idea to add more resources than the minimum one, also considering the container needs.

Find the right support.

You can install Kubernetes from scratch, or using the Ansible playbook directly on your preferred VMs. Anyway, looking to the official K8S documentation, we will find these two good alternatives:

CloudStack, it is based on hardware virtualization principles, like traditional IaaS, and it is available also for on-prem solutions and supports Vagrant
CoreOS is a good alternative for deploying the Kubernetes cluster. You can do it manually using this tutorial or using the CloudStack integration.

The Team

Kubernetes is not a game. It is quite easy to install (just note that it is shipped with docker) and to use for deploying the application. The part that is a little bit more complex is the management. I do not mean just look that the resources are enough and, in case, add a new node. I’m speaking about ensure backup of volumes, distribute services for balancing the load over nodes, or simply troubleshoot problems.

Kubernetes is the perfect tool because, in most scenarios, you just need to install an then work alone. Meanwhile, in the cloud solution part of the architecture is managed, on-premise is all up to you. This is why I suggest to each company that is adopting it to have almost one trained specialist. The exam from CNCF is an excellent point to start from. There are many cheap preparation courses you can buy on Udemy or similar platforms.

Don’t forget about DevOps

Kubernetes’ adoption is not just a matter of Kubernetes. You need all the architecture behind him. The standard development flow with Kubernetes requires a significant change in the IT mindset. This is also translated in tools (yes, other tools…) you have to provide to developers. First of all, you will need a CR (container repository) to store all container images, and these images will be produced by a DevOps process, implemented by a tool. If your company doesn’t allow the cloud for security reasons, you will need to deploy such a tool locally. In practice, Kubernetes bring to a well-structured process. If you haven’t yet you will be forced to bring it at home.

What to take home

The cloud is the first choice for Kubernetes unless you are in a particular situation where you have specific benefits like cost optimization, hard requirements like security, or some particular condition that require full control on all the stack. If you are in these cases, you will need to provide a reliable infrastructure based on solid hardware and in a good installation. Still, you will also need people trained in the product and ready to manage the Kubernetes complexity.

The good news is that you can move (quite) easily in the future if the on-prem will be too tight-fitting for your need.

A note about CoreOS

May 26 is coming and the official Kubernetes documentation is pointing to Cloudstack and CoreOS (the last update was in June 2019 ), but that distribution is reaching end-of-life on May 26th. The successor designed by Red Hats is Fedora CoreOS, that’s okay for starting a new project, but needs some work in case of migration. In that case is an option to use the Flatcar Container Linux distribution, which is a more compatible solution. Another opportunity is to move to the cloud, if already in plans. This end of life could be the right nudge.