10 Dec 2015 in Research, Analysis, Docker, Containers

The State of Containers and the Future of the Docker Ecosystem

Containers (and in particular, Docker) are getting ever more popular.

A recent report by O’Reilly Media and Ruxit presents interesting findings on the adoption and use patterns of containers and Docker.

For instance: the deployment of containers in production is likely to increase significantly in the short term. The report also highlights that one of the major barriers preventing production adoption has to do with the need for better operations tools. This sort of information may be crucial in guiding decision making on investment and innovation priorities.

This post considers some key aspects of the report. I first present the approach used for the research, then highlight the main findings. I conclude with a quick comparison to similar research reports published during the course of this year.

Read More
8 Dec 2015 in CoreOS, Containers, Fleet, Etcd

Run Self-Sufficient Containers on CoreOS

In my last post we learnt about CoreOS and took a look at the steps needed to install it on your laptop (using VirtualBox). We also learnt CoreOS doesn’t ship a package manager. Instead it comes with Docker pre-installed. So, for every service that you need (e.g. web server, database, cache, and so on) you can just create and use a Docker container for it.

So, what’s the deal with self-sufficient containers?

Containers are self-sufficient by default, right? Well, this depends on what you call self-sufficient. Of course, containers are self-sufficient in that they don’t depend on any software running outside of their logical boundaries. But they need kernel support and computing resources from the host computer.

Let’s see it like this: containers are generally set up on a network with several nodes, with each node running one or more containers. Each container may be self-sufficient, but the node on which the container is supposed to run is not! Nodes may crash or run slow or get disconnected off the network. And in such cases, you need to find out the offline node and re-route the traffic meant for that node to other nodes.

But how do we do that?

Enter CoreOS and three particularly crucial components: systemd, fleet, and etcd.

In this post, we’ll take a look at these three components and learn how they’re used by CoreOS to help you create self-sufficient containers.

Read More
3 Dec 2015 in Announcement, Helm, Kubernetes

Helm 0.2.0 Released

Since KubeCon 2015, the Helm team has been hard at work bringing you a new release.

Helm 0.2.0 contains numerous bug fixes, some code refactoring, and several enticing new features.

Highlights:

  • Helm has been moved to github.com/helm/helm and the core charts are now at github.com/helm/charts
  • helm lint helps you validate your charts.
  • helm repo add|rm|list lets you easily manage your own chart repositories.
  • Support for Kubernetes 1.1 beta 1 kinds (DaemonSet, Job, etc.) as well as custom kinds.
  • helm install and helm uninstall are smarter! In fact, many commands are improved.
  • Helm now supports git-style plugins.
  • Many, many bug fixes.

We're already hard at work on Helm 0.3 and have some exciting new features in the works.

Follow along on our GitHub Milestone for 0.3.0.

1 Dec 2015 in Series: Schedulers, Docker, Fleet, Swarm, Overview

Schedulers, Part 1: Basic Monolithic

Scheduling is a method to to assign workloads to resources that can handle those workloads. In a distributed environments, there is a particularly important need for schedulers. Especially ones that provide scalability, are resource aware, and cost effective.

Monolithic schedulers are a single process entity that make scheduling decisions and deploy jobs to be scheduled. These jobs could be a long running server, a short living batch command, a MapReduce query, and so on.

For monolithic scheduler to make decision for scheduling a job, it should observe the resources available in the cluster (e.g. CPU, memory, and so on), lock the resources, schedule the job, and update the available resources.

It’s hard for Monolithic schedulers to deal with more than one job at a time because there is a single resource manager entity a single scheduling entity. Avoiding concurrency makes the system easier to design and easier to understand.

Some examples of monolithic schedulers:

  • fleet: a native scheduler to CoreOS (not resource aware)
  • Swarm: a scheduling backend for Docker containers
  • Kubernetes: an advanced type of monolithic scheduler for Pods (a collection of co-located containers that share same namespaces)

In this post, we'll cover two basic monolithic schedulers.

First we'll look fleet, a scheduler for systemd units. systemd is an advanced init system that that configures and boots Linux userspace. A unit is an individual systemd configuration file that describes a process you’d like to run. We’ll also look at Swarm, a scheduling backend for Docker containers.

Read More
27 Nov 2015 in Deis v1 PaaS, LTS, Community Meeting

Deis 1.12.2 Release and Latest Release Planning

Deis v1.12.2 Release

Deis v1.12.2 will be released on Tuesday, December 1. We have a variety of bug fixes in this release, and the team continues to make progress on the upcoming version 2 of Deis. Please refer to the upgrade documentation to assist in your cluster migration.

A full list of items in this release will be provided on December 1.

Read More