17 Oct 2016
This is the first Workflow team developer update! In this blog post we'll give an update on how
things have been rolling on the Workflow team for the past few weeks, what has been our main focus
areas and some non-technical updates on how the team has been doing.
People Come, People Go
Over the past few weeks, we've had some amazingly talented people join our team, and equally amazing
people move onto bigger and better things.
Matt Tucker has joined the Workflow team as a Software Engineer
and Core Maintainer of Workflow. He has experience both working with Go and with Kubernetes,
migrating code to production on Kubernetes supporting many Drupal and Wordpress sites. Welcome to
the team, Matt!
On a sadder note, Helgi Þorbjörnsson has decided to move on. He is still
part of the open source community and is still a maintainer, and he is available on the #community
channel in Slack and he still pops by on the occasional Github issue. We will miss you, Helgi!
Joshua Anderson was one of our (returning) summer interns, and
has since returned back to school for another year to continue his studies for computer engineering
at California Polytechnic State University.
While not technically leaving us, Jack Francis has shifted over to
the Helm team to do great things with Helm and Kubernetes package management!
The Great Helm Migration
Our team is heavy at work testing the migration strategy from the old Helm (known as "Helm Classic")
to the new Helm. So far, we have deployed charts for every Workflow component. Look out for the
charts directory in each repository!
For more history on Helm and Helm Classic, Matt Butcher wrote a wonderfully insightful blog post on
Helm's First Birthday.
CI Stability Improvements
Our Continuous Integration servers have been continuously (heh) improving! A few weeks ago our CI
system was very unreliable when it came to testing new features or running end-to-end test suites
on a live cluster. Thanks to many stability improvements from GKE and from our own stability
improvements to the end-to-end test suite, the
controller release job now has a success rate of 93%, and the
only test failure occurred because
DockerHub was down at that point. This is a
significant win over the v1 acceptance test suite, which is well-known at this point to be
unreliable due to internal networking
issues, provisioning issues with Vagrant or other unknown general mayhem.
Test Coverage Improvements
Some great leaps and bounds have been made to cover more ground on the test coverage side. Just
because we live in a Kubernetes-centric world and some components are tightly coupled does not mean
we can't write good code! In the last few releases, both the builder and the router have seen a test
coverage increase of at least 15% in total coverage, bringing them both up to at least 55% coverage.
The Workflow CLI has also been brought up to 72% coverage
in the past few weeks. Test coverage has improved greatly, but there's still more room for
If you're feeling motivated, read the docs and make a pull
...Or as we like to call it - "drinking our own champagne" - we are now running a Workflow cluster
internally for dogfooding purposes. We have a couple applications running on the cluster already:
- deis-bot, a mention bot for Github comments and pull requests
- jenkins-ci, another mention bot for notifying specific channels when a build or end-to-end test fails
- k8s-claimer, a tool we use to lease out a cluster for the CI system
All of these components are running on this cluster, and we intend to run more applications such as
our main website on it at some point.
One of the biggest incubator projects we got going on right now is Steward. Steward is a
Kubernetes-native service broker. Modeled after the
Cloud Foundry Service Broker System, it
functions as a gateway from your cluster-aware applications to other services, both inside and
outside your cluster.
For those of you aware of my endless posts about us finally getting to developing a Service Broker
to provision and bind apps to databases, external filesystems, and other third-party systems like
Heroku's Marketplace, this is it.
Specifically, its high-level goals are to:
- Decouple the provider of the service from its consumers
- Allow operators to independently scale and manage applications and the services they depend on
- Provide a standard way for operators to:
- Publish a catalog of services to operators or other interested parties
- Provision a service
- Bind an application to a service
- Configure the application to consume the service through standard Kubernetes resources
As of right now, Alpha 1 has been
released. The project is still in a pre-production state, but we're working on getting everything in
place before a v1.0.0 release.
With all that said, here's a summary on some of the things we plan on doing over the coming weeks:
- a full migration story from Helm Classic to Helm
- make individual Workflow components more useful to the broader Kubernetes community
- more apps on our dogfooding cluster
- continue to improve Steward
- continue to improve test coverage
Thanks for reading the first Workflow developer update! Please feel free to reach out to me on Slack
or on twitter if you feel like you enjoyed this update, what we
can do to improve upon this post or general comments/questions about something. Thanks!