Hi there!
Today we’re going to share about Jenkins X.
I think every developer knows what Jenkins is. Or at least has heard of it once. But what is ‘X’?
Basically, Jenkins X is not just a CI/CD tool to run your builds and deployments, it is an attempt to automate the whole development process end to end for containerised applications based on Docker and Kubernetes. It is obviously open source, as all best applications.
This sounds very promising at a high level. So, let’s see what’s behind the glam cover of Jenkins X!
TL;DR:
Here is our takeaway from the Jenkins X project:
Though project has been made public just recently, it appears to be mature enough to make use of it in Production systems.
Although migration of existing CI/CD pipelines into Jenkins X might be found difficult due to existing custom processes, new projects may benefit from using Jenkins X a lot, as it saves significant amount of effort by setting up basic GitOps flow out of the box.
Customisation of the Jenkins X system shouldn’t be a problem as it utilizes already familiar tools like Jenkins, Make, Docker, Helm
Strengths of Jenkins X:
Not that good points of Jenkins X:
Our opinion on Jenkins X is that it goes into right direction, addressing the pain points and streamlining implementation of GitOps principles. So it’s effectively DevOps for everyone, providing basic, standardised implementation, though there is still a room for customization.
Most probably, we’ll see more radical implementations of similar tools in future, making PaaS implementations even easier, but less customizable. However, this is the common technological trend: deliver things simpler, faster, cheaper.
Ok, if you want to get to know Jenkins X closer, let’s jump into details!
The main problem, which Jenkins X (‘JX’ hereafter) is going to address is formulated by its creators as ‘make it simple for developers to work to DevOps principles and best practices’. Some of that are:
In plain English, that means JX takes a challenge to automate a creation of development environments and facilitate development process. So, once JX is installed and set up, it does:
In order to achieve the goal and solve the problem set, JX makes use of 4 main building blocks:
As different components of the system utilise different vocabulary, Jenkins X tries to span across all domains with its own abstraction layer. And the most important entities of that one include the following items:
There is much more entities than that in JX, but the ones above are essential for understanding the concept of JX.
Now, that we know the main building blocks, let’s try to put them together. Next picture shows how parts of JX system are connected in static:
Figure 1. Components of the system
As you can see, associations between different entities of the components are pretty straight forward: JX abstractions are mapped 1:1 to Git, k8s and Helm objects.
To get more understanding of how everything works, It’s worth to consider each of the components in more details:
Kubernetes is the foundation of a development infrastructure. Its ecosystem covers most of the software development and operational needs in modern IT world. Distributed cluster manages Docker containers and provides features like auto-scaling, self-healing, secrets and configuration management, automatic rollout/rollback and others.
For more about Kubernetes, please check out following articles:
https://blog.octo.com/en/how-does-it-work-kubernetes-episode-1-kubernetes-general-architecture/
https://blog.octo.com/en/the-twelve-factors-kubernetes/
Kubernetes hosts all services deployed by JX, including administrative ones (Jenkins, Chartmuseum, Monocular etc). Deployment of the services (or applications) is coordinated via Helm. Helm’s Charts allow sharing of application templates and makes versioning easy. Helm also takes care of upgrade and rollback cases, which makes it quite useful.
Each Helm Chart is a folder, that contains the following files:
As part of JX installation, following services are deployed to facilitate Helm Charts managements:
Jenkins instance is deployed to Kubernetes cluster as part of the installation of JX. It runs in “master/agent” distributed mode. That means there is always a ‘master’ process running, serving requests to Jenkins console and distributing the load to multiple ‘agents’. This allows to process multiple pipeline runs in parallel and evenly distribute the load, create/delete agent nodes based on demand.
Master and each of the agents are deployed as separate Pods in Kubernetes, this can be easily monitored via kubectl CLI or a dashboard.
Jenkins pipelines are typically described in a configuration file called Jenkinsfile. This file contains set of stages to execute for each pipeline. It’s normally checked into VCS along with an application code.
In order to fulfil GitOps requirements, Jx uses Git to store two types of data:
So, JX is connecting all these pieces together with its smartly weaved fabric of automation scripts and configurations. And the pattern of that fabric is known as GitOps**.**
What is GitOps? This is a set of principles for managing software and infrastructure based on Git:
As you can see, there is a direct association between JX Environments, Git Repos and k8s Namespaces. Actually, JX environment is nothing more than just an abstraction, which keeps configuration in specific Git repo and deploys to the corresponding k8s namespace.
Applications (in our case it’s myGreatApp) are another abstraction in JX. For each of the applications, corresponding Git repo is created. Different versions of apps can be deployed (promoted) to different environments in k8s, but only through changing of the configuration of particular environment in Git.
Now, we’re coming to the main element of JX, the tool which does all the magic, glueing all building blocks together and providing an entry point to a system management and orchestration. It’s a JX CLI interface and it’s written in Go. The JX CLI is utilized by end users to manage resources (apps, environments, urls etc) as well as in Jenkins pipelines created by JX. Some of the most important commands are provided below:
As a next step, let’s consider the lifecycle of the application development, from the initiation and all the way to the Production deployment. At a high level, it’s presented on the diagram below:
The picture above shows, that JX covers not only operational processes based on Git, but also provides the ways of initiating an application, as well as promoting to Production, thus covering e2e cycle.
Let’s have a look at how the JX processes correlate to Git actions. For simplicity, let’s use only master and one feature branch. The table details the steps of the flow.
Figure 2. Git workflow events
Alright, so once we’ve covered got some overview of JX, let’s see what happens when we install it! But before that, the main prerequisite for JX installation is to have Kubernetes cluster (or Minikube - simplified single-node Kubernetes cluster) installed, as well as command-line Kubernetes CLI - Kubectl and lastly - VM environment if we choose to go with Minikube (either VirtualBox, VMware Fusion, or HyperKit). There is a lot of things happening with our Kubernetes cluster in background of the installation. Below is a diagram showing the process of what JX does:
Figure 3. JenkinsX moved functionality
Once JX is installed, it initialises Pods with the following applications in jx namespace of k8s:
This jx namespace of k8s is associated with the Dev environment in JX. Those host all utility tools which are used for development, but are not being developed themselves.
Apart from Pod installation, JX creates staging and production environments that hold versions of applications being developed. These environments configurations are pushed to remote Git repos and webhook is created to link to Jenkins installation in k8s.
It is worth to mention that JX exposes IPs of some Pods via Ingress. Which means you can access all the utility tools like Docker Registry, Monocular or Nexus from an external to k8s network.
JX supports the creation of a Quickstart application based on Node JS, Spring boot, GoLang or Rust (as of now) . This is basically a creation of application structure from template. When quickstart is created, JX initializes a local repository, unpacks code template, adds the Jenkinsfile template, Helm charts, Dockerfile and Makefile from so called ‘draft’, and then pushes the new repo to the provided remote Git.
Figure 4. Jenkins X quickstart installation and first pipeline run
As a part of quickstart installation, JX initializes the first pipeline as following:
Considering the type of application, if any artefacts are created (such as Spring boot app), they will be pushed to Nexus when a new version of app is created/promoted.
Even though JX positions itself as a sub project of the Jenkins Foundation, it has a very vibrant and responsive community. During our initial product exploration, we opened issues on GitHub page regarding JX functionality. Surprisingly, major contributors resolved our issues within a couple of hours with immediate actions. We are very pleased with such a fast feedback.
Figure 5. Example of GitHub issue discussion
Jenkins X spans quite a number of technologies and though it is intended to make application development lifecycle easier, it still requires an understanding of many technical concepts.
Apart from the complexity of the toolset, Jenkins X appears to be a promising project which can level up a DevOps experience on Kubernetes and reduce initial implementation costs.
Its Quickstarts and configurable drafts can fulfil the customisation needs of picky DevOps engineers allowing (with some additional effort) to go beyond basic GitOps flow and standard application templates.
The well articulated roadmap shows the width of intentions and instills confidence in the ability of Jenkins X to cover a variety of technical cases.
It would be nice to see more detailed documentation on main principles and architecture of the system, as well as technical details on some CLI commands. There is also very limited information available on making custom extensions like Quickstarts and drafts.
| name | Description |
| Git | The most popular VCS |
| Jenkins | An open source CI / CD automation platform |
| Jenkins X | A subproject of Jenkins, automates CI/CD processes in Kubernetes based systems |
| Kubernetes | a.k.a. k8s, open source, distributed management system of containerized applications |
| Minikube | A tool to run Kubernetes locally |
| GitOps | A set of principles for managing software and infrastructure based on Git |
| Docker | A containers management platform |
| Docker registry | A repository to store and distribute Docker containers |
| Nexus | An artifact repository |
| Helm | A package manager for Kubernetes |
| Helm Chart | A collection of configuration files that describe a related set of Kubernetes resources. E.g. chart, describing an application stack |
| Chartmuseum | An open-source Helm Chart Repository |
| Monocular | An open-source web-based UI for Helm Chart management |
| Quickstart | Pre-made Jenkins X applications templates a developer can start a project from, instead of starting from scratch |
Authors downunder:
Ilya Trofimov and Nick Shulhin