Finding The Right Local Kubernetes Development Environment
In the Kubernetes world, there's no single source of truth and rarely can you find a single recommended best-practice. This is nice (diversity!) but can also get quite confusing if you're just getting started.
There are many local development environments out there, so in this post, I'll take a look at a few of them and give you a bit of help choosing one that works for you.
Disclaimer: This is in no way a complete guide. I have limited it to just three common cases. If you think I've overlooked an important option, just shoot me a message and I might add it to the list.
CoreOS Kubernetes With Vagrant
This approach uses CoreOS VMs, and setup done for you, including TLS encryption between components as well as DNS and (optionally) even Calico networking.
You can configure things, like how many nodes with how much RAM you want, and which version of CoreOS should be running.
Kube Solo and Cluster for OS X
Both tools are based on xhyve and thus supports Mac OS X only. However, neither tool requires Virutal Box nor Vagrant. Kube Solo is for single node setups, and Kube Cluster is for multi node setups.
Like the Vagrant approach, both tools are based on CoreOS and fleet as an underlying base layer and come with a running DNS setup.
Kube Solo and Kube Cluster both come with UIs for fleet and Kubernetes. There's even a nice menubar icon, giving you access to all kinds of commands and settings. For example, updating Kubernetes or CoreOS.
Kubernetes in Docker With Hyperkube
Last but not least, you can just run Kubernetes in Docker, which works pretty much anywhere Docker works.
There are several pre-built Docker images and a few guides that can help you with this. Arguably, however, the most used and most updated (being built and pushed automatically with every release of Kubernetes) is hyperkube.
Check out the official documentation for more info on setting stuff up.
Unfortunately, the docs are quite long, and you will need to perform many steps to get your setup to a state comparable to the previously mentioned solutions. For example, you need to deploy DNS manually, which can be tricky if you're new to Kubernetes.
Check out the dev_k8s shell script by Charlie Drage if you want a quick way of getting set up with hyperkube.
Decisions, Decisions, Decisions
Picking a solution depends on four things:
- Your host system
- What you'd like to test out
- How much flexibility you want to have
- How fast and easy it should be
Your Host System
If you have Mac OS X, the xhyve based tools (Kube Solo and Kube Cluster) are the more native solution.
Docker moving towards an xhyve based local environment in the current Docker for Mac beta shows there's a trend towards more native virtualization on Mac OS.
If you don't run Mac OS X, your choice is limited to Docker or Vagrant by default.
If you want an environment for your whole organization or team, going with one that runs on any system might be the best idea.
What You'd Like to Test Out
If you want to test high availablity, you must choose a cluster (multi node) solution.
If you like to test out different networking solutions or other "under the hood" extensions consider how much flexibility you want. Which brings me to...
How Much Flexibility You Want to Have
Changing stuff under the hood is usually easier when there's a well documented system like CoreOS and fleet running below it. And Vagrant boxes give you a bit more control as it's easier to fiddle with them.
If you want to test out different networking solutions, like Calico or (soon) Canal, the CoreOS setup will help because it comes with Calico on board. It can also be used to test other networking solutions, like Weave.
How Fast and Easy It Should Be
The xhyve and Docker based solutions are generally faster (to startup) than VM based solutions like Vagrant.
The Kube Solo and Kube Cluster is as easy as it this stuff is going to get. They're definitely the nicest from a UX perspective.
I personally use Kube Solo and Kube Cluster for most things, but go for a Vagrant setup when I want more flexibility and need to work more deeply within the system.
However, if you already have Docker installed and would prefer not to install anything else, you can get a quick first glipmse of Kubernetes with hyperkube.
Whichever you choose, once Kubernetes is up and running, you'll probably want to start playing with things.
There are plenty of Kubernetes tutorials, but finding the right one and not feeling overwhelmed by the huge amount of new concepts is not easy. One good way to combat this is to familiarise yourself with the basic concepts before you dive in.
Check out Rimantas Mocevicius's (the author of Kube Solo and Kube Cluster) miniseries giving an overview of Kubernetes.