CoreOS is the Operating system designed for Docker containers. CoreOS has Docker pre-installed out of the box. Kubernetes may be installed on CoreOS using CloudFormation, as discussed in detail in an earlier article, “Getting Started with Kubernetes on Amazon Web Services (AWS).”
Problem
Kubernetes is not pre-installed on CoreOS and may be installed by using the kube-aws tool and it requires some user input and configuration. Installing Kubernetes on CoreOS is an involved process and consists of the following stages shown in Figure 1.
Figure 1: Kubernetes installation on CoreOS Stages
Solution
A Jenkins Pipeline may be used to automate the installation of Kubernetes, as shown in Figure 2. The various stages of installation may be configured in a Jenkinsfile and, when the Pipeline is run, the kube-aws tool gets downloaded, the CloudFormation stack gets initialized, the contents of the Asset Directory get rendered, the cluster parameters such as number of worker instances get customized, and subsequently the cluster gets validated and launched.
Figure 2: Jenkins Pipeline for installing Kubernetes on CoreOS
Installing Kubernetes using a Jenkins Pipeline is an example of the Automation DevOps Design Pattern.
In a three-article tutorial, we shall automate the Kubernetes installation process using a Jenkins Pipeline. This article has the following sections:
Setting the Environment
We shall install Jenkins using Docker image “jenkins” on a CoreOS instance. We shall launch a Kubernetes cluster on an Amazon AMI Linux EC2 instance using a Jenkins Pipeline. First, launch two EC2 instances, one running Amazon Linux and the other running CoreOS, as shown in Figure 3.
Figure 3: Obtaining the Public IP Address
When creating the Amazon Linux instance, create a new key pair (“jenkins”, for example) by selecting “Create a new key pair” and download the private key “jenkins.pem” to be used for configuring a Jenkins agent. Obtain the Public IP Address of the EC2 instance running CoreOS and SSH log in to the instance.
ssh -i "jenkins.pem" core@52.90.166.25
Run the Docker image for Jenkins to launch Jenkins.
docker run -name jenkins -p 8080:8080 -p 50000:50000 jenkins
Docker image “Jenkins” gets downloaded. Jenkins gets started. Copy the password generated. Obtain the Public DNS of the EC2 instance running Jenkins. Using the url <public dns>:8080, log in to the Jenkins Dashboard, as shown in Figure 4.
Figure 4: Jenkins Admin Console
Creating the Pre-requisite Artifacts
Now, we need to create some artifacts for a Kubernetes cluster that are not amenable to automation.
- EC2 key-pair
- KMS key
- Jenkins directory /var/jenkins
- SSH login to the Amazon EC2 instance running Amazon Linux
ssh -i "jenkins.pem" ec2-user@ec2-54-161-44-160.compute.1.amazonaws.com
Because Amazon EC2 is used, an AWS account is required. We need to create a set of AWS Security credentials, which we shall use to configure the EC2 instance from which the CloudFormation stack is launched. To create new AWS Security credentials, click Security Credentials for the user account and click Create New Access Key to create an access key. Copy the Access Key Id and the access key. In the Amazon Linux instance, run the following command to configure the instance with the AWS credentials:
aws configure
Specify the access key id and access key when prompted. Specify the default region name (us-east-1) and the output format (json), as shown in Figure 5.
Figure 5: Configuring Jenkins Instance with AWS Credentials, Region, and Default Output Format
Next, create a EC2 key-pair. Run the following command to create a key pair called kubernetes-coreos and save it as kubernetes-coreos.pem.
aws ec2 create-key-pair --key-name kubernetes-coreos --query 'KeyMaterial' --output text > kubernetes-coreos.pem
Modify the access permissions of the key pair using the mode as 400, which sets access permissions to read by owner.
chmod 400 kubernetes-coreos.pem
A key pair gets created and access permissions get set (see Figure 6).
Figure 6: Creating a Key Pair for Kubernetes
The key pair also gets listed in the AWS EC2 Console, as shown in Figure 7.
Figure 7: Obtaining the Public IP Address
Next, create a KMS key, which is used to encrypt/decrypt cluster TLS assets and is identified by an Arn string. Use the aws command line interface to create a KMS key for region us-east-1.
aws kms --region=us-east-1 create-key --description="kube-aws assets"
A KMS key gets created, as shown in Figure 8. Copy the KeyMetadata.Arn string starting with arn:aws:kms:us-east-1 to be used later to initialize the cluster CloudFormation.
Figure 8: Creating a KMS Key
We also need to create a directory for Jenkins:
sudo mkdir /var/Jenkins sudo chmod 777 /var/jenkins
A directory gets created and permissions get set, as shown in Figure 9.
Figure 9: Creating a Directory for Jenkins
Creating a Jenkins Node
A Jenkins Pipeline is a Jenkins project that makes use of the Jenkins Pipeline plug-in. A pipeline typically consists of a sequence of steps, with each step performing a task. Available executors on agents are used to run a Jenkins project. An “agent” is a machine configured to offload projects from the master node. The “master” node is the machine on which Jenkins is installed and handles all the tasks for the build system, including parsing Pipeline scripts and running a Jenkins project on an executor. An “executor” is a computational resource for compiling code and it could be configured on the master or agent nodes. In this section, we shall create an agent node and configure an executor on the node. Select Manage Jenkins in the Jenkins Dashboard, as shown in Figure 10.
Figure 10: Obtaining the Public IP Address
Subsequently, select Manage Nodes, as shown in Figure 11.
Figure 11: Obtaining the Public IP Address
The “master” node should get listed. Click New Node to create an agent node, as shown in Figure 12.
Figure 12: Obtaining the Public IP Address
Specify a Node name (jenkins, for example) and select the Permanent Agent radio button, as shown in Figure 13. Click OK.
Figure 13: Obtaining the Public IP Address
To configure the new agent node, we shall need the host DNS on which the agent is to be created. Copy the Public DNS from the AWS EC2 Console for the EC2 instance running an Amazon Linux image, as shown in Figure 14.
Figure 14: Obtaining the Public IP Address
In the new agent input user interface, specify a Name (jenkins, for example). Specify the # of executors as 1. Specify a Remote root directory as /var/jenkins, which was created earlier. Specify Labels as “jenkins”, whose significance we shall discuss in a subsequent section. In Usage, keep the default setting of “Use this node as much as possible.” In Launch method, select “Launch slave agents on Unix machines via SSH.” In Host, specify the Public DNS copied from the EC2 Console. In Availability, select “Keep this agent online as much as possible.” For Credentials, click the Add drop-down and select Jenkins Credentials Provider, as shown in Figure 15.
Figure 15: Obtaining the Public IP Address
In the Add Credentials dialog, select Domain as Global credentials and Kind as Global. Specify Username as “ec2-user” and select Enter directly. Copy and paste the contents of the private key file jenkins.pem in the Key field, as shown in Figure 16.
Figure 16: Adding Jenkins Credentials
Click Add, as shown in Figure 17.
Figure 17: Jenkins Credentials Provider>Add
Select the ec2-user credential in Credentials, as shown in Figure 18. Click Save.
Figure 18: Configuring Jenkins Node
A new agent gets added, as shown in Figure 19. Initially, the agent may not be available because it is still getting started.
Figure 19: Jenkins Agent created
Click the agent link and click Relaunch agent, if required, as shown in Figure 20.
Figure 20: Re-launching Agent
The output message “Agent successfully connected and online” indicates the agent has been launched (see Figure 21).
Figure 21: Agent successfully connected and online
The “jenkins” agent should get listed as running, as shown in Figure 22.
Figure 22: Jenkins Agent Running
Conclusion
In this article, we introduced the importance of automating a Kubernetes installation with Jenkins. We got started by installing Jenkins on CoreOS, creating the pre-requisite artifacts, and creating a Jenkins node. In a subsequent article, we shall configure a Jenkinsfile for a Jenkins pipeline and create a Jenkins pipeline. And, in a 3rd article, we shall run the Jenkins pipeline to install Kubernetes.
