Storage in PaaS: Minio and Deis Workflow
Whether you notice it or not—as an end user—storage is an important component of almost all the software we use today. As a developer however, it is important is to be able retrieve stuff in an easy yet secure and fast way.
In this post, we'll see how Deis, an open source PaaS based on Kubernetes, uses Minio for almost all of its storage requirements.
But first, introductions.
Simply put, Deis is a platform that lets developers easily deploy and manage applications.
By easily, I mean developers can deploy their code with a simple
git push. Deis handles all the building, packaging and deployment processes.
With Deis, developers can also manage the application configuration, create or roll-back releases, get extensive logging, and more.
Right now, there are two Deis flavors available. First is Deis Workflow (Deis v2.0) based on Kubernetes. Second is Deis v1.13.0, based on CoreOS. It is a long term support release and there are no further features planned for this line.
Deis Workflow is the successor of Deis, and hence, we'll limit our discussion to Deis Workflow.
Minio is a minimalistic object storage server written in Golang.
The Minio server, client, and SDK are API compatible with Amazon S3 cloud storage service.
If you're wondering what all these elements are for:
- The Minio server lets you store unstructured data files in a safe, reliable and easy to retrive way
- The Minio client is S3 compatible implementation of common file handling commands like ls, cp, mkdir, diff and more
- The Minio SDK lets you embed Minio APIs in your own applications
For any queries about Minio, head to the Gitter channel. We're pretty active there and will love to help you out!
Minio in Deis
Deis Workflow is a group of self-contained components that communicate using both the Kubernetes system and an object storage server.
There are three deployment methods available in Deis:
- Buildpack deployment: deploy apps based on Heroku buildpacks. With this method, you can deploy applications already running on Heroku or applications that follow Heroku best practices.
- Dockerfile deployment: deploy apps via Dockerfiles. Push the code and a Dockerfile creates the Docker image to be deployed.
- Docker image deployment: deploy apps via an existing Docker image. To do this, you'll have to create the image locally and push it to a public Docker image registry (e.g. Docker Hub). Then use
deis pullto deploy the application from the registry directly.
All three deployment methods (as well as many of the Deis Workflow internals) use Minio extensively behind the scenes:
- Buildpack deploys use Minio to store code and slugs
- Dockerfile deploys use Minio to store Dockerfiles and associated artifacts
- Docker image deploys use Minio as the backing store for the internal Docker registry that Workflow runs
- The internal Workflow database backs all data up to Minio
Minio & Deis Workflow in Action
To see Minio in action while using Deis Workflow, you'll need to have Deis Workflow installed. When Deis Workflow has all its pods running, you can see Minio pod among them. This pod is responsible for backup of critical user and application data.
Let's take it for a spin.
Install Helm Classic and Deis Workflow CLI Tools
The Deis Workflow CLI lets you interact with Deis Workflow.
Ideally, you should install the client in your
$PATH to avoid any
bash: command not found errors. So, change your directory to
/usr/local/bin and then install the Deis Workflow client by running:
Check if it's working by running:
Next you'll need to install Helm Classic. Helm is a package manager for Kubernetes. We'll use it to manage software in our Kubernetes cluster. Install it by running:
Check the install is successful:
Boot a Kubernetes Cluster
Next step is to boot up a Kubernetes cluster.
I have used AWS EC2 as the backend cloud. You can choose to use Google Container Engine (GKE) or even your own laptop.
Since we'll use command line interface to interact with AWS, you'll need to install the AWS and Kubernetes CLI tools.
Once you're done, run:
Now add the Key ID, Secret Access Key, default region name, and the output format.
You can now interact with your AWS account via the CLI!
Create a folder at a suitable place. This will serve as the home to your Kubernetes installation. Then download and unzip Kubernetes.
Configure the Kubernetes environment:
Boot up the cluster by running:
Install Deis Workflow
To install Deis Workflow, we'll need to use Helm.
First check if it can connect with your Kubernetes cluster:
You should get a detailed list like this:
This confirms that Helm can talk to your Kubernetes cluster.
The next step is to add Deis charts repo to Helm:
Then install Deis Workflow:
This will complete the installation process.
You'll have to wait a few minutes for the pods to be ready. You can check the progress with this command:
Once all the pods show a ready status, Deis Workflow is up and running.
Here's what that will look like:
Deis Workflow can be thought of as a collection several pods, with each doing a predefined activity. Here you can see various pods in action.
You can see the
deis-minio pod running, confirming that Minio is the default object storage used by Deis.
In this post, we took a brief look at both Deis Workflow and Minio. Deis Workflow is an application platform for Kubernetes and Minio is a minimalistic object storage server that is used by Deis. We installed Deis Workflow and spun up a cluster and saw that Minio is right there waiting for use.
If you want to deploy an app on your new cluster, follow this quick start.
This post originally appeared on the Minio blog.