The Workflow "Umbrella" Helm Chart

3 Jan 2017

Last month's v2.9.0 release of Workflow included officially transitioning over to Kubernetes Helm. Here we look at how our CI system assembles the Workflow chart, which is itself an "umbrella" chart consisting of sub-charts for each of its core components.

What is an umbrella chart?

Just as applications of any complexity tend to consist of smaller, "building block" components, an umbrella chart is built up of multiple sub-charts brought together to form a larger whole.

The umbrella chart can then supply global configuration to these sub-charts by way of the top level values.yaml file. The required version for each sub-chart is specified in the aptly-named requirements.yaml file.

In fact, thanks to Keerthan Mala's contribution, each version field in the aforementioned file also allows for a semver range to be supplied, if an exact match is not desired.

How CI assembles the Workflow chart

For Workflow, each component chart is defined and managed in its own GitHub repository. Packaging the Workflow chart gets even trickier when a different variant is desired depending on environment and particular CI pipeline. These environments come in the form of chart repos (AWS S3 buckets at time of writing), one each for pull requests, master builds, release candidates and official releases.

Component chart CI

As an example, let's look at the CI flow for pull requests against a component (sub-)chart (the flow is also very similar for merges to the master branch in each component, assuming the chart is affected):

  1. The first job checks out the PR commit, sets the version to a patch-incremented, pre-release value such as v1.2.4-<timestamp>-sha.<short_sha> (where the current official component release would be v1.2.3) and then packages and uploads the chart to the component's pr chart repo.

  2. The next job packages the umbrella Workflow chart after setting said component's chart version to v1.2.4-<timestamp>-sha.<short_sha> and chart repo to the pr variant in the requirements.yaml. (The rest of the component versions won't be specified; instead, chart repos for all will be the dev variant and helm dependency update will simply supply the latest chart in said repo.) The assembled chart is then uploaded to Workflow's pr chart repo.

  3. Lastly, a downstream E2E job is triggered, which will install the Workflow chart, using the exact version handed to it, onto a "leased" k8s cluster from a pool of ready-to-go GKE clusters we maintain. The Workflow cluster, assuming it stands up successfully, is then vetted against the current E2E test suite. The test results are archived and the job status is reported to the GitHub Pull Request.

Workflow chart CI

Another CI flow that may be informative is how we stage and release the Workflow chart:

  1. The first job, workflow-chart-stage, is kicked off with the intended chart version, which should match the semver release string that will be used to tag the repo. It will:

    • Request the latest (git) release for all component repos (which align with their respective release chart versions)
    • Set the versions appropriately in requirements.yaml, make sure all chart repos are the production variants and run helm dependency update
    • Sign and package the workflow chart and upload to the production chart repo sans index.yaml -- therefore, the signed release artifact is ready for official release if all testing gives the thumbs up; all that needs to be done is upload the updated index.yaml to make it available
    • After altering a few values in values.yaml to point to their non-production counterparts, again package the workflow chart, this time uploading to the staging chart repo
  2. E2E is run against the chart from the staging repo (and manual testing is executed)

  3. Assuming the release candidate chart in staging is looking good, the last job is workflow-chart-release, which simply does the aforementioned update and upload of the index.yaml to the production chart repo, thereby allowing users to fetch and install the official release chart.

  4. Lastly, a job is triggered to run helm fetch --verify on the workflow chart just released, to make sure there are no issues with verification of provenance.

Wrapping Up

Hopefully the examples presented above offer some ideas on how CI pipelines can support umbrella Helm chart packaging and releasing. All of the CI code mentioned above can be found in the deis/jenkins-jobs repo.

Have you also had to tackle similar with your Helm charts? If so, we'd love to hear about your approach. Holler at us on the #community channel on Slack.

Posted in Community Meeting, Deis Workflow

triangle square circle

Did you enjoy this post?