Tag: Workflow
17 Oct 2016 in Workflow, Developer Update

Workflow Developer Update

This is the first Workflow team developer update! In this blog post we'll give an update on how things have been rolling on the Workflow team for the past few weeks, what has been our main focus areas and some non-technical updates on how the team has been doing.

People Come, People Go

Over the past few weeks, we've had some amazingly talented people join our team, and equally amazing people move onto bigger and better things.

Matt Tucker has joined the Workflow team as a Software Engineer and Core Maintainer of Workflow. He has experience both working with Go and with Kubernetes, migrating code to production on Kubernetes supporting many Drupal and Wordpress sites. Welcome to the team, Matt!

On a sadder note, Helgi Þorbjörnsson has decided to move on. He is still part of the open source community and is still a maintainer, and he is available on the #community channel in Slack and he still pops by on the occasional Github issue. We will miss you, Helgi!

Joshua Anderson was one of our (returning) summer interns, and has since returned back to school for another year to continue his studies for computer engineering at California Polytechnic State University.

While not technically leaving us, Jack Francis has shifted over to the Helm team to do great things with Helm and Kubernetes package management!

The Great Helm Migration

Our team is heavy at work testing the migration strategy from the old Helm (known as "Helm Classic") to the new Helm. So far, we have deployed charts for every Workflow component. Look out for the charts directory in each repository!

For more history on Helm and Helm Classic, Matt Butcher wrote a wonderfully insightful blog post on Helm's First Birthday.

CI Stability Improvements

Our Continuous Integration servers have been continuously (heh) improving! A few weeks ago our CI system was very unreliable when it came to testing new features or running end-to-end test suites on a live cluster. Thanks to many stability improvements from GKE and from our own stability improvements to the end-to-end test suite, the controller release job now has a success rate of 93%, and the only test failure occurred because DockerHub was down at that point. This is a significant win over the v1 acceptance test suite, which is well-known at this point to be unreliable due to internal networking issues, provisioning issues with Vagrant or other unknown general mayhem.

Test Coverage Improvements

Some great leaps and bounds have been made to cover more ground on the test coverage side. Just because we live in a Kubernetes-centric world and some components are tightly coupled does not mean we can't write good code! In the last few releases, both the builder and the router have seen a test coverage increase of at least 15% in total coverage, bringing them both up to at least 55% coverage. The Workflow CLI has also been brought up to 72% coverage in the past few weeks. Test coverage has improved greatly, but there's still more room for improvement.

If you're feeling motivated, read the docs and make a pull request!


...Or as we like to call it - "drinking our own champagne" - we are now running a Workflow cluster internally for dogfooding purposes. We have a couple applications running on the cluster already:

  • deis-bot, a mention bot for Github comments and pull requests
  • jenkins-ci, another mention bot for notifying specific channels when a build or end-to-end test fails
  • k8s-claimer, a tool we use to lease out a cluster for the CI system

All of these components are running on this cluster, and we intend to run more applications such as our main website on it at some point.


One of the biggest incubator projects we got going on right now is Steward. Steward is a Kubernetes-native service broker. Modeled after the Cloud Foundry Service Broker System, it functions as a gateway from your cluster-aware applications to other services, both inside and outside your cluster.

For those of you aware of my endless posts about us finally getting to developing a Service Broker to provision and bind apps to databases, external filesystems, and other third-party systems like Heroku's Marketplace, this is it.

Specifically, its high-level goals are to:

  1. Decouple the provider of the service from its consumers
  2. Allow operators to independently scale and manage applications and the services they depend on
  3. Provide a standard way for operators to:
    • Publish a catalog of services to operators or other interested parties
    • Provision a service
    • Bind an application to a service
    • Configure the application to consume the service through standard Kubernetes resources

As of right now, Alpha 1 has been released. The project is still in a pre-production state, but we're working on getting everything in place before a v1.0.0 release.

What's Next?

With all that said, here's a summary on some of the things we plan on doing over the coming weeks:

  • a full migration story from Helm Classic to Helm
  • make individual Workflow components more useful to the broader Kubernetes community
  • more apps on our dogfooding cluster
  • continue to improve Steward
  • continue to improve test coverage

Thanks for reading the first Workflow developer update! Please feel free to reach out to me on Slack or on twitter if you feel like you enjoyed this update, what we can do to improve upon this post or general comments/questions about something. Thanks!

4 Oct 2016 in Workflow, GKE, Series: Workflow on GKE

Production Deis Workflow on Google Container Engine, Part Two

This is part two of a two part series that walks you through a full production setup of Deis Workflow.

In part one, we set up off-cluster object storage, a Docker registry, and a Postgres platform database. We then installed Workflow on a Kubernetes cluster.

In this post, I will show you how to secure your cluster with SSL and get DNS set up for your Workflow domain. Finally, I will show you how to upgrade Workflow itself.

Read More
3 Oct 2016 in Workflow, Release, Announcement

Deis Workflow 2.6 Release

Summer may have come to a close in September, but that doesn't mean the Workflow train stops rolling.

The team recently cut Workflow 2.6 which contains a lot of bug fixes and package bumps. Listen to the Pomplamoose cover of Earth Wind and Fire's "September" and cruise through the highlights.

Release Highlights

InfluxDB and Telegraf have been bumped to the 1.0 versions.

Workflow now adds additional metadata to the application environment. This is useful for debugging application environments, can help with auditing, and might be something you want to include in an application health response.

We've cut new base images across the board which include fixes for the recent OpenSSL CVEs.

The logger component went to summer camp and cleaned up its act and is no longer using excessive cpu.

Thanks to community member @robholland, router will now pass through X-Request-Id and X-Correlation-Id headers.

Router also saw a fix for builder + PROXY_PROTOCOL contributed by @felixbuenemann.

Default buildpack versions have been bumped across the board:

There are many smaller fixes, cleanups and documentation additions. Check out the complete Workflow 2.6 change logs in the docs.

Up Next

Our next release is scheduled for October 11th, 2016. You can check out the 2.7 milestone on each of the component repositories, or take a gander at the Workflow Roadmap.

October Community Meeting

The Deis Community Meeting is Thursday October 6th. Join us at 11am Pacific/1900 UTC to talk about what we've been up to. Add the meeting to your calendar or join us directly on Zoom at https://engineyard.zoom.us/j/510595782.

23 Sep 2016 in Workflow, GKE, Series: Workflow on GKE

Production Deis Workflow on Google Container Engine, Part One

Deis Workflow has been in GA for a while. But what's it like to work with? Particularly, how do we set it up for production use?

In this two-part miniseries, I'll walk you through a full production setup of Deis Workflow.

This will include:

  • How to set up off-cluster object storage, Docker registry, and Postgres database
  • How to install Workflow on a Kubernetes cluster
  • How to secure it with SSL and set up DNS for your Workflow domain
  • How to upgrade Workflow to the latest release

Let's jump right in.

Read More
9 Sep 2016 in Workflow, Release, Announcement

Deis Workflow 2.5 Release

The best way to roll into the weekend is with fresh software, hot off the presses. The Deis Workflow team just merged the final charts for 2.5!

We've got a ton of functionality packed into 2.5, so hold on to your horses!

Workflow 2.5 includes initial support for Kubernetes Horizontal Pod Autoscaling. Which is not only a mouthful, but pretty neat to boot. Workflow 2.5's theme song is "Glassworks" by Philip Glass. I'm pretty sure this is what a Horizontal Pod Autoscaler would sound like if it made noise.

Cast scale at the darkness...

Setting a scaling policy for your application is straightforward. Policies are set per process-type, which allows developers to easily scale processes independently:

$ deis autoscale:set web --min=3 --max=8 --cpu-percent=75
Applying autoscale settings for process type web on scenic-icehouse... done

The Kubernetes HorizontalPodAutoscaler (HPA) does require CPU limits to be set for the application process type, so makes sure you set a limit:

$ deis limits:set web=250m -c
Applying limits... done

=== scenic-icehouse Limits

--- Memory
web     Unlimited

--- CPU
web     250m

Behind the scenes, the HorizontalPodAutoscaler (HPA) will spring into action, adding or removing pods so that the average CPU utilization of your application processes approach the CPU target.

There is a bit of nuance to the way HPAs work so spend a bit of time with the Kubernetes documentation on the algorithm.

Viewing and removing scaling policies are simple CLI commands as well:

$ deis autoscale:list
=== scenic-icehouse Autoscale

--- web:
Min Replicas: 3
Max Replicas: 8
CPU: 75%

$ deis autoscale:unset web
Removing autoscale for process type web on scenic-icehouse... done

Autoscaling in Workflow should be considered Alpha and we would love your feedback!

Build Result Caching

Thanks to community member @jeroenvisser101 the Workflow build system now caches build results. This change greatly speeds up the process on subsequent builds.

Enforce SSL Application by Application

Workflow 2.5 now allows developers to require TLS on a per-application basis. Instead of a global setting in the router (via router.deis.io/nginx.ssl.enforce), Workflow CLI has a few new tricks:

$ deis tls:enable -a spicy-icehouse
Enabling https-only requests for spicy-icehouse... done

Now, connections on port 80 for this application will be be redirected with HTTP status code 301 to the HTTPS version. Since this interaction occurs at the edge router, developers aren't required to use application middleware to enforce HTTPS.

To allow both HTTP and HTTPS traffic for an application (which is the default) use tls:disable:

$ deis tls:disable -a foo
Disabling https-only requests for foo... done

Application-specific IP Address Whitelisting

Developers and operators who need to control access to applications by IP address, Workflow 2.5 makes this process much easier!

Using the CLI developers may manage IP whitelisting per application:

$ deis whitelist:add, -a drafty-zaniness
Adding, to drafty-zaniness whitelist...done

$ deis whitelist:remove -a drafty-zaniness
Removing from drafty-zaniness whitelist... done

$ deis whitelist -a drafty-zaniness
=== drafty-zaniness Whitelisted Addresses

Adding a whitelist to an application automatically rejects connections from any un-listed address. Removing the last IP address from a whitelist returns the application to the default behavior, which accepts connections from any IP address.

Full Release Changelogs

Workflow 2.5 changes are now available in the Workflow documentation. No more crawling through GitHub repositories or past blog posts to learn about changes.

Up Next

Our next release is scheduled for September 28th, 2016. You can check out the 2.6 milestone on each of the component repositories, or take a gander at the Workflow Roadmap.

29 Aug 2016 in Workflow, Stackpoint Cloud

Stackpoint Cloud and Deis Workflow

Stackpoint Cloud provides a simple and easy to use interface to provision Kubernetes on a variety of clouds. Whether you are on Amazon, Azure, Digital Ocean, or Packet, Stackpoint Cloud is a great way to get started with Kubernetes.

We announced a new collaboration at the beginning of August. Not slowing down, Stackpoint has added support for Kubernetes 1.3.5 and Deis Workflow 2.4.

This guide shows just how easy it is to bring up a Stackpoint Kubernetes cluster with Deis Workflow automatically installed and configured.

Stackpoint and Workflow Overview

Read More
24 Aug 2016 in Workflow

Private Applications on Workflow

Last week, we released Workflow v2.4.

In Workflow v2.4, we added something called deis routing. This feature allows you to add or remove an application from the routing layer. If you remove an application from the routing layer, it continues to run within the cluster while being unreachable from the outside world.

If you remove an application from the routing layer, the application is still reachable internally thanks to Kubernetes services. This allows for some pretty neat interactions where users can run internal service APIs or backing services like postgres without exposing it to the outside world.

In this post, I'll take a closer look at this new feature and show you how and why you'd want to use it with your application.

Read More
22 Aug 2016 in monitoring, Workflow, Sysdig

Monitoring Deis Workflow with Sysdig Cloud

Deis Workflow is an open source Platform as a Service that works with and compliments Kubernetes. This is great if you want a Heroku-inspired workflow for easily deploying your container-based apps to the cloud.

But what if you want to monitor them as well and get insight into app performance? That's where Sysdig Cloud comes in.

Sysdig Cloud is "container-native monitoring". It comes with built in integration for many popular server software products, gives you a realtime dashboard, historical replay, dynamic topology mapping, and alerting. It also natively integrates with Kubernetes so that you can monitor your services as well as your containers and hosts.

So let's combine these two products!

In this post I am going to show you how to monitor Deis Workflow with Sysdig Cloud.

Read More
19 Aug 2016 in Workflow, Release, Announcement

Deis Workflow 2.4 Release

The Deis Workflow team continues to ship great features every two weeks. I am starting to run out of pithy intros for these release announcements. So I'll let DEVO take it from here.

When a feature comes along, you must ship it.

Read More
3 Aug 2016 in Workflow, Release, Announcement

Deis Workflow 2.3 Release

Another two weeks, another Workflow release. The 2.3 release brings with it some internal release changes, improved private registry support, tools to call for help, and faster deploys. You might say Papa's got a brand new bag.

Read More
25 Jul 2016 in Workflow, Release, Announcement

Deis Workflow 2.2 Release

It may be the hottest July on record but we've got some new cool stuff in the latest release of Deis Workflow. Sit back, grab a cold drink, and check out the latest Workflow release.

Read More
8 Jul 2016 in Storage, Minio, Workflow

Storage in PaaS: Minio and Deis Workflow

Whether you notice it or not—as an end user—storage is an important component of almost all the software we use today. As a developer however, it is important is to be able retrieve stuff in an easy yet secure and fast way.

As I have mentioned elsewhere, object storage is a great way to achieve this. I've also previously looked at how to create a reliable data store, taking WordPress as an example.

In this post, we'll see how Deis, an open source PaaS based on Kubernetes, uses Minio for almost all of its storage requirements.

But first, introductions.

Read More
5 Jul 2016 in Workflow, Release, Announcement

Deis Workflow 2.1 Release

Happy Tuesday, I hope everyone had a wonderful weekend! Before we struck out for fun in the weekend sun we cut a hot and fresh release of Workflow. Arriving as version 2.1 we've got lots of fixes and a few goodies to boot.

Read More
9 Jun 2016 in Workflow, Release, Announcement

Deis Workflow, Now Stable!

I am proud to announce the first stable release of Deis Workflow. This means the Deis community now considers Workflow suitable for production deployments. Deis Workflow is the first PaaS built on Kubernetes to reach this milestone.

Deis Workflow is the new name for our open source PaaS, and is the second major version of what we are now calling Deis v1.

According to Docker Hub, Deis v1 has been downloaded over 6.5 million times.

Deis v1 is trusted in production by hundreds of companies, including Mozilla, The RealReal, Hearst Corporation, and dozens of others.

What did we change in version two?

Well, the most significant thing we did was re-platform from CoreOS Fleet to Kubernetes. The switch to Kubernetes gives us a stable cluster manager, a better scheduler, a smaller overall footprint, and a great Kubernetes community to work with.

Read More