Deis Workflow 2.2 Release
It may be the hottest July on record but we've got some new cool stuff in the latest release of Deis Workflow. Sit back, grab a cold drink, and check out the latest Workflow release.
Now Powered by Deployments
As part of Deis Workflow 2.2, developers can enable the use of Kubernetes
deis config:set DEIS_KUBERNETES_DEPLOYMENTS=1 will use
start using Deployments under the hood. No need to re-create your application.
Previous to this release, Workflow would orchestrate rolling updates just like
kubectl used to. Workflow's controller would increment and decrement replica
counts over both old and new Replication Controllers. By using Deployments,
all of this orchestration happens within Kubernetes control loops. This has
some pretty sweet advantages:
- deis/controller code is simplified
- server-side execution means deployments will complete even when client connections are broken (internet drops, ctrl + c, etc)
- release and rollbacks are also tracked in Kubernetes, giving operators total visibility
- this opens the door for HorizontalPodAutoscaling in a future release
Cluster operators that would like Deployments as their default methodology
today can set
DEIS_KUBERNETES_DEPLOYMENTS=1 environment variable in the
Changing deployment mechanisms can always reveal subtle timing behaviors that application developers may have inadvertently come to rely on. So we are rolling this out on an opt-in, per-application basis.
We hope to make this the default behavior in a few releases, so kick the tires and let us know if anything behaves out of the ordinary.
We've continued to refine the log shipping system and now include an instance
of Redis. This allows us to horizontally scale the number of
processors while maintaining a consistent view via
We've also added a new SumoLogic plugin to our
fluentd setup. For more
information on how to configure fluentd to ship to more backends see the
New Metric Dashboards
You may not know it, but Workflow ships a full metrics collection stack designed to keep an eye on all the Workflow components. The stack is backed by Fluentd, Telegraf, InfluxDB, and Grafana and monitors platform and component health. We will continue to add detailed component metrics over time. For information on the architecture and details on accessing the dashboard see the platform monitoring documentation:
Workflow Component Health
This dashboard aggregates memory and CPU usage for all Deis Workflow components.
Router metrics gives you a high-level view of the routing subsystem including aggregate status codes, per-application request rate, per-application and aggregate response times.
The InfluxDB dashboard lets you keep an eye on the rate at which series are flowing into the database.
Kubernetes APIServer Health
The Kubernetes dashboard brings together API server, kubelet, and Docker engine response times. Quickly see if any nodes are starting to act up.
NSQ dashboard shows queue depths for both logs and metrics. As well as a real-time count of metric and log consumers.
Redis now holds a rolling set of logs per-application. This dashboard gives you overall redis health including CPU, memory and total operations per second.
Upgrading to Workflow 2.2
Upgrading is very easy if you've configured off-cluster object storage. Follow our upgrade instructions and enjoy Workflow 2.2.
Highlighted Changes in Workflow 2.2
b59bbbc(fluentd) - fluentd: Adding sumologic plugin support
424523c(logger) - storage: Add redis storage adapter
2da72a5(logger) - redis: Optimize with more aggresive pipelining
0c82466(logger) - storage: Make redis the default storage adapter
2f92eca(monitor) - telegraf, grafana: Start collecting redis metrics
c9718e4(charts) - logger: Add redis instance for use by logger
7d40069(charts) - swift: add support for swift storage
d6992e1(charts) - telegraf: Configure telegraf to start fetching redis metrics
0ae9d90(workflow-cli) - deis: add
2862f05(workflow-cli) - colors: reserve magenta for controller log messages (#132)
8a61e63(controller) - scheduler: add support for set based requirement filtering via the kubernetes API
47b5b08(controller) - IDN: add support for international domains
32be50a(controller) - scheduler: sort env vars and secrets by keys for easier hashing
c3c2494(controller) - add Deployments support behind a feature flag
edb0383(controller) - scheduler: feat(scheduler) prepend [namespace] to Scheduler log message for better traceability
84b8080(controller) - app: make deploy timeout configurable globally/per-app via DEISDEPLOYTIMEOUT, default is 2 minutes
65f8714(dockerbuilder) - objectstore: set properly builder bucket file environment variable
9179923(dockerbuilder) - deploy.py: handle chunked output errors
02112ed(dockerbuilder) - deploy.py: delay so stderr is logged before pod exits
93b9c5c(logger) - redis: Pass style checks
7144c4e(logger) - rc: Specify use of redis storage adapter
9dae9cc(monitor) - telegraf: Create nsq topic at startup
a9b2275(monitor) - telegraf: Update image default image tag in manifests
8fd5ada(monitor) - grafana: Fix blank dashboards from appearing in dropdown
7de06cc(monitor) - grafana: Redis cpu graph was not selecting right data
0492bf8(charts) - database: Fix logic for selecting off-cluster db
6fd8407(charts) - logger: Do not helm keep rc deis-logger-redis
c425baf(workflow-cli) - ps: restarting a single pod created by Deployments was not working
bcb8b01(workflow-cli) - ps: give ps:restart a better understanding between RC and Deployment pods
4b2c6d4(workflow-cli) - scale: dont call controller if there is no valid scale pattern
32177aa(workflow-cli) - settings: don't panic on empty settings file (#134)
6214d96(controller) - scheduler: if one RC fails to scale then ensure all other RCs are at the right level
0bcea13(controller) - scale: return error message in proper format
7d24923(controller) - restart: wait for the pods to be scheduled
7647569(controller) - scheduler: cast port to an int from environment (#857)
b27c816(controller) - boot: change group ownership of docker socket to deis (#804)
d4415c9(controller) - api: fail when rolling back to v1 (#762)
1ac6b54(controller) - release: return port from get_port for non-routable process types
a0567ea(controller) - api: remove command escaping from v1 (#822)
f403efb(controller) - tests: sort domains in tests to get past occasional ordering problems which cause test failures
dd2b505(workflow) - logging,monitoring: Update platform-logging and monitoring docs with recent changes
dfb6c8e(workflow) - install-workflow: note that k8s 1.2.x is required
aa40a49(workflow) - swift: Add swift as an object storage
3e7e865(workflow) - deployments: add Deployments documentation
ee03d28(workflow) - apps: document
DEIS_DEPLOY_TIMEOUTand the nuances around that
5c289c4(slugbuilder) - buildpacks: update heroku-buildpack-go to v42
d5cdd0b(slugbuilder) - buildpacks: update heroku-buildpack-php to v108
85c8292(slugbuilder) - buildpacks: update heroku-buildpack-nodejs to v91
165105f(slugbuilder) - buildpacks: update heroku-buildpack-python to v81
9bbed87(slugbuilder) - buildpacks: update heroku-buildpack-scala to v71