1 Dec 2015 in Series: Schedulers, Docker, Fleet, Swarm, Overview

Schedulers, Part 1: Basic Monolithic

Scheduling is a method to to assign workloads to resources that can handle those workloads. In a distributed environments, there is a particularly important need for schedulers. Especially ones that provide scalability, are resource aware, and cost effective.

Monolithic schedulers are a single process entity that make scheduling decisions and deploy jobs to be scheduled. These jobs could be a long running server, a short living batch command, a MapReduce query, and so on.

For monolithic scheduler to make decision for scheduling a job, it should observe the resources available in the cluster (e.g. CPU, memory, and so on), lock the resources, schedule the job, and update the available resources.

It’s hard for Monolithic schedulers to deal with more than one job at a time because there is a single resource manager entity a single scheduling entity. Avoiding concurrency makes the system easier to design and easier to understand.

Some examples of monolithic schedulers:

  • fleet: a native scheduler to CoreOS (not resource aware)
  • Swarm: a scheduling backend for Docker containers
  • Kubernetes: an advanced type of monolithic scheduler for Pods (a collection of co-located containers that share same namespaces)

In this post, we'll cover two basic monolithic schedulers.

First we'll look fleet, a scheduler for systemd units. systemd is an advanced init system that that configures and boots Linux userspace. A unit is an individual systemd configuration file that describes a process you’d like to run. We’ll also look at Swarm, a scheduling backend for Docker containers.

Read More
27 Nov 2015 in Deis v1 PaaS, LTS, Community Meeting

Deis 1.12.2 Release and Latest Release Planning

Deis v1.12.2 Release

Deis v1.12.2 will be released on Tuesday, December 1. We have a variety of bug fixes in this release, and the team continues to make progress on the upcoming version 2 of Deis. Please refer to the upgrade documentation to assist in your cluster migration.

A full list of items in this release will be provided on December 1.

Read More
24 Nov 2015 in Announcement

Star Power

Wow! Deis passed 5,000 stars on GitHub and I couldn't be more happy.

I wanted to take this moment to say THANK YOU to our amazing community.

You're the real stars! ;)

19 Nov 2015 in Docker, Docker Hub

Six Ready-made NoSQL Database Docker Images

NoSQL is an umbrella term for a whole category of databases, many different in their featuresets, but united in that they depart in some way from the relational model used by databases such as MySQL and PostgreSQL.

In this post, we cover six ready-made NoSQL database Docker images.

For each image, we address some of the current bugs, and offer potential workarounds.

Rethink DB

RethinkDB is an OSS project built to store JSON documents and has been designed with horizontal scalability in mind. RethinkDB supports functions, table joins, aggregations, and groupings to bolster its native query language, ReQL.

ReQL differs from other query languages in that, rather than constructing strings for a query engine, developers work with ReQL via chainable methods directly from their programming language of choice. As well as being a easy to learn, this also mitigates against the posibility of injection attacks.

Read More
17 Nov 2015 in Helm, Kubernetes

Why Kubernetes Needs Helm

Earlier this week, Deis released Helm—the package manager for Kubernetes. Conceptually similar to Homebrew, Helm makes it easy to install standard workloads on Kubernetes.<!--more-->

But... Kubernetes is a container platform, so why does it need a package manager?

Perhaps looking at OS-level package managers (like Homebrew, apt, yum/rpm, ports, and so on) will help explain the situation.

Why use apt, yum, or homebrew?

Let's take a typical scenario.

I'm sat at the terminal in the chilly server room. I tried the command again: ./configure && make. Page after page of information scrolled across the screen. Apache httpd was building. I flipped open my book to read a few pages while I waited. Several minutes later, I saw the make command fail. I just wanted a stock Apache httpd server, but I couldn't figure the right combination of build flags, nor could I find and install all of the correct dependencies.

In frustration, I gave up and tried the radical approach: I switched operating systems.

Then when the time came to install Apache httpd, I simply typed apt-get install apache. And hey presto! It worked. If I needed to make changes, I could head to the /etc/httpd directory and configure away. But even prior to that I had a functioning web server. Apache httpd was working right out of the box.

Read More