Introduction

In a Jenkins environment, the Jenkins Master (Server) is the central component that manages and orchestrates all the workflows defined in the pipelines. It is responsible for scheduling jobs, monitoring their progress, and ensuring that they are executed efficiently. The Jenkins Master is essentially a control server that coordinates all the activities of the Jenkins environment. On the other hand, a Jenkins Agent is a machine or container that connects to the Jenkins Master and executes the steps mentioned in a job.

When a Jenkins job is created, an agent must be assigned to it. Each agent has a unique identifier, known as a label, that distinguishes it from other agents. When a Jenkins job is triggered from the master, the actual execution occurs on the agent node configured in the job. The agent node is responsible for carrying out the tasks specified in the job, such as building software, running tests, and deploying applications.

In a small team with a relatively small number of projects, a single, monolithic Jenkins installation can be sufficient. However, as the needs of the team grow, it often becomes necessary to scale up the Jenkins environment. This is where the "master-to-agent connection" comes in. In a master-to-agent connection, the Jenkins Master continues to serve the Jenkins UI and act as a control node, while the agents handle the execution of jobs. This allows the Jenkins environment to scale more efficiently, as the agents can be distributed across multiple machines or containers, enabling the environment to handle a larger workload.

In this setup, the Jenkins Master is responsible for scheduling jobs, monitoring their progress, and ensuring that they are executed efficiently. The agents, on the other hand, are responsible for executing the jobs that are assigned to them by the master. The use of agents in a Jenkins environment provides several benefits, including:

Scalability: By distributing the execution of jobs across multiple agents, the Jenkins environment can handle a larger workload, making it more scalable.
Flexibility: Agents can be configured to execute jobs in different environments, such as different operating systems, hardware configurations, or software configurations.
Efficiency: Agents can be optimized for specific tasks, such as building software or running tests, which can improve the efficiency of the Jenkins environment.
Reduced load on the master: By offloading the execution of jobs to agents, the load on the Jenkins Master is reduced, which can improve its performance and extend its lifespan.

To conclude, the Jenkins Master is the central component of a Jenkins environment that manages and orchestrates all the workflows defined in the pipelines. The Jenkins Agent is a machine or container that connects to the Jenkins Master and executes the steps mentioned in a job. In a master-to-agent connection, the Jenkins Master continues to serve the Jenkins UI and act as a control node, while the agents handle the execution of jobs, providing scalability, flexibility, efficiency, and reduced load on the master.

Pre-requisites

Let's say we are creating a new EC2 instance server to act as an agent, we need to install Java on the given machine as Jenkins itself is based on Java and it is required to execute tasks. It's not necessary to install Jenkins on this machine.

Task-01

Create an agent by setting up a node on Jenkins
Create a new AWS EC2 Instance and connect it to the master(Where Jenkins is installed)
The connection of the master and agent requires SSH and the public-private key pair exchange.
Verify its status under the "Nodes" section.

The first logical step will be to create a new EC2 instance that is going to act as a slave for my Jenkins master. Usually in production, there is a separate master Jenkins server.

Now, updating the packages and installing Java in the new server

sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version

Make sure there are the same Java versions in both instances.

It is time to create a connection between both instances, the master and the slave using SSH.

Generate RSA Key Pair on Master Server

Access the terminal on your Jenkins master server.
Navigate to the .ssh directory: cd ~/.ssh
Generate an RSA key pair: ssh-keygen
Follow the prompts to name the key pair (e.g., id_rsa) and set an optional passphrase. This passphrase adds an extra layer of security.
Verify the key generation: ls

Copy the key.

Add Master Server's Public Key to Slave Server

Navigate to the .ssh directory on the slave server: cd ~/.ssh
Open the authorized_keys file using a text editor: nano authorized_keys
Paste the master server's public key on a new line.
Save the changes and exit the text editor.

Connect to Slave from Master Server

In the terminal on the master server, use SSH to connect to the slave server: ssh <slave_username>@<slave_server_address> or ssh slave_public_IPaddress

Congratulations we have established a connection between two systems using SSH.

Now let's set up the agent on the master Jenkins server.

Setup a new node

Do specify the remote root directory.

The Host should be the public IP address of the slave/agent. And as we are connecting through SSH select the below option.

The host key Verification strategy that we are going to use is through Non verifying Verification Strategy.

For credentials enter the SSH details here. For reference refer to the below pictures.

Here we need to insert the private SSH key from the master node that we created so that the connection is established.

Lets see and try to launch our agent.

Note: Sometimes there may be trouble in launching an agent.
Some points which can help you :
Try to restart the Jenkins

Launch new ssh-keygen -t ed25519 on the master and try to follow the same steps.

Task-02

Run your previous Jobs (which you built on Day 26, and Day 27) on the new agent
Use labels for the agent, your master server should trigger builds for the agent server.

We will run this Hello-world program using our created agent.