Continuous deployment of a webserver with zero downtime using Jenkins, Kubernetes, and Docker with E2E automation.

In this article, I am going to show “Continuous deployment of a webserver with zero downtime using Jenkins, Kubernetes, and Docker with E2E automation.”

PREREQUISITE:

1. RHEL 8 (GUI/CLI)
2. Docker-CE

3. Jenkins

4. Kubernetes (Should be configured on Windows and RHEL)

5. Git

This project was to fully integrate GitHub, Jenkins, Docker to deploy data to a web server. We have also used Kubernetes to manage the Pods.

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early. By integrating regularly, you can detect errors quickly, and locate them more easily.

In this project, we have successfully created a CI/CD pipeline by integrating GitHub, Jenkins, Docker and we will deploy them as pods on Kubernetes by creating deployments, replica set, services, and make their data persistent. We have launched the jobs in Jenkins with help of Jenkins File inside which the code is written in Groovy language and it enables the programmatic creation of Jenkins jobs using Groovy code. The automatic creation of various jobs using the Jenkins file happens due to Jenkins Job DSL for which you need to download Job DSL Plugin. Job DSL was one of the first popular plugins for Jenkins which allows managing configuration as code. In Jenkins we have created 4 Jobs which are being created using seed job and are responsible for various tasks.

We will start by creating a Docker file in which kubectl will be configured. We will launch Kubernetes on top of Docker and the deployments will be created with persistent storage. We will copy “web-pv.yml”, “web-pvc.yml” and “web-deploy.yml” in the right path of our docker image. We will also copy “ca.crt”, “client.crt” and “client.key” which will be required for the configuration of kubectl on our docker image.

Plugins required to be installed before proceeding to the task.

1. Job DSL
2. Script Security Plugin
3. Workspace cleanup
4. Docker plugin
5. Email Extension
6. GitHub Plugin

The Job DSL plugin helps you to seed your Jenkins jobs from a DSL file. Out-of-the-box Jenkins does not support DSL file. To enable it you will need to install the Job DSL plugin.

Now we create SEED JOB

The seed job is a Jenkins job that runs a DSL script and then generates a new job. The seed job is a normal freestyle Jenkins job that you add the “Process job DSL” build step. This step takes the DSL and generates the configured jobs.

You can refer the GitHub repository where DSL script is uploaded.

pchat99/devops_project3

Now when we build a seed job. The output looks like this.

GENERATED JOB1:

You can refer to “Jenkins Job DSL Plugin” for built-in DSL methods. The built-in Job DSL currently supports 184 Jenkins plugins.

The docker file and “web-pv.yml”, “web-pvc.yml” and “web-deploy.yml” used in this task are the same as we have used in CI/CD using Jenkins and Kubernetes. You can refer to the GitHub Repository.

We need to allow the Docker daemon to be accessed remotely. Since docker is isolated. For that we need to make changes in /usr/lib/systemd/system/docker.service.

Here 0.0.0.0 means that you can denote it with any IP given to the DOCKER HOST and you can assign this IP with any available port.

After this process, Restart the services

1. systemctl daemon-reload
2. systemctl restart docker.service

Now we need to export DOCKER_HOST. Use command

1. export DOCKER_HOST=<your IP address>:<port you assigned>
2. echo $DOCKER_HOST

Now to setup Docker cloud on Jenkins. Go to the following section.

Manage Jenkins > Manage Nodes and Clouds > Configure Clouds > Add a new cloud

Now we will add the Docker template in the Docker Cloud settings. In the Docker template, we need to give a label which we will use during the configuration of Job 2 and we need to give the right path of our Docker file which has kubectl configured. In container settings, we need to give the right path of the config file which is present in your RHEL.

Container Settings > Volumes

/root/task4:/root/.kube/

**Right side is the path where we have mounted config file that is to be used in Docker container. The left side is the path where we have mounted the config file in RHEL.

Config file used in Docker file for kubectl.

GENERATED JOB 2:

JOB 2 OUTPUT:

We will launch Kubernetes on top of Docker and the deployments will be created with persistent storage. We will copy “web-pv.yml”, “web-pvc.yml” and “web-deploy.yml” in the right path of our docker image. We will also copy “ca.crt” , “client.crt” and “client.key” which will be required for the configuration of kubectl on our docker image. This job has to run on dynamic container slave for that we have already added a template on Docker cloud. It will automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes. This job will create a new deployment (if it does not exist) or else we will roll out updates to the existing deployment. Since we our building this job for the first time, So it will create a new deployment. To create a Persistent Volume(PV), persistentVolumeClaim(PVC), and deployment we will be using YAML files “web-pv.yml”, “web-pvc.yml” and “web-deploy.yml”. We will also expose the deployment.

The web page is also running fine.

The monitoring will be handled by Kubernetes itself.

Now to send an E-mail notification from Jenkins. We need to do some pre-requisite before sending an E-mail to the developer, I will be using Gmail as the mail server here,

1. Go to Google account settings
2. Turn on the “Less secure app access”
3. Turn off 2-Step Verification

Now we have to make changes in the Jenkins configuration file.

Open “vim /etc/sysconfig/jenkins” and edit JENKINS_JAVA_OPTION

Restart the Jenkins service

systemctl restart Jenkins

Now we need to configure the E-mail plugin

Go to Manage Jenkins — Configure System

If you are using TLS, then use port 587. If you are using SSL, then use port 465

SMTP server: Server to be used for mailing service. I have used Gmail here.

Test it and run it. Before building a job.

GENERATED JOB 3:

GENERATED JOB 4:

When there is some error in the HTML code or Pods crash due to some error. It will send an e-mail to the developer.

As you can see our web page is deployed properly. if anything goes wrong in a pod running and it crashes. Kubernetes will launch another pod on its own.

Project completed successfully under the guidance of Vimal Daga Sir.