Jenkins Fundamentals - Part 1: What Are We Actually Operating?

When someone first learns Jenkins, the workflow often looks deceptively simple: a developer pushes code, Jenkins runs, and the build either succeeds or fails.

That view is useful at the beginning, but it hides most of the concepts that matter once Jenkins becomes part of a real engineering platform. Between the moment Jenkins receives a build request and the moment a build command actually starts running, several components are involved: the Controller, Queue, execution node, Executor, Workspace, and persistent Jenkins state.

Before we install Jenkins on AWS, add build workers, introduce ephemeral agents, or discuss production architecture, we first need a precise mental model of what is actually happening inside Jenkins.

That is the goal of this article.

1. Jenkins Is More Than a Build Server

At a high level, Jenkins is an automation server that orchestrates CI/CD workloads. A more useful way to think about it, however, is to separate Jenkins into two responsibilities: control and orchestration, and workload execution.

The Jenkins Controller is responsible for coordinating the system. The execution environment is responsible for running the actual build commands.

A simplified Jenkins environment can be visualized like this:

In a small installation, the Controller itself may also act as the execution node. That is convenient for learning because everything runs on one machine, but it also mixes orchestration and build workloads together.

Later in the series, we will separate those responsibilities. For now, the important thing is to understand the building blocks.

2. Jenkins Controller

The Jenkins Controller is the central coordination point of a Jenkins environment. It serves the Jenkins Web UI and API, receives build triggers, maintains the Queue, schedules workloads, coordinates Pipeline execution, manages Agents, and stores Jenkins configuration and build metadata.

The Controller decides what should run, when it should run, and where it should run.

That does not mean the Controller has to perform the actual compilation, testing, scanning, packaging, or deployment itself. Those workloads can be delegated to Jenkins Agents.

This separation between orchestration and execution is one of the most important architectural ideas in Jenkins. It allows the platform to evolve from a simple all-in-one installation into a distributed system without changing the fundamental execution model.

For example, the architecture may evolve in several steps.

• We may begin with the Controller and build workloads sharing a single EC2 instance:

• Then move to a dedicated Controller with multiple static EC2 Agents.

• Later, we can replace static workers with ephemeral Agents.

The infrastructure changes, but the Controller remains responsible for orchestration.

3. Jenkins Agent

A Jenkins Agent is an execution environment that performs work assigned by the Controller. This is where commands such as compilation, testing, security scanning, packaging, or deployment usually take place.

Depending on the pipeline, an Agent might run commands such as:

An Agent can be implemented in many ways. It may be a long-running EC2 instance, a virtual machine in another cloud, a Kubernetes Pod, a Docker container, an ECS Fargate Task, or even a physical server.

From Jenkins' point of view, these are all execution nodes capable of running workloads. Agents also make workload separation possible. For example, a Java build can use an environment with Java and Maven installed, while an infrastructure pipeline can use a different Agent with Terraform and the AWS CLI.

Agent: java-builder Tools: - Java - Maven - Gradle Agent: node-builder Tools: - Node.js - npm Agent: terraform Tools: - Terraform - AWS CLI 

We will build a static Jenkins Agent on Amazon EC2 in a later article. For now, the key distinction is simple: Controller = orchestration, while Agent = execution environment.

4. Executor: An Execution Slot

One of the easiest Jenkins concepts to misunderstand is the Executor.

An Agent and an Executor are not the same thing. The Agent is the compute environment, while the Executor represents a slot on that environment that Jenkins can use to run a workload.

Suppose an EC2 Agent is configured with two executors:

That node can potentially execute two Jenkins workloads concurrently. If both Executors are busy and another compatible build arrives, that build has to wait in the Queue.

This is where the Jenkins Queue becomes important.

5. Jenkins Queue

Receiving a build request does not necessarily mean Jenkins starts executing it immediately. The request may first enter the Jenkins Queue while the Controller waits for a suitable execution environment and an available Executor.

A build may remain in the Queue because all compatible Executors are busy, the required Agent is offline, the Pipeline requests a label that no available node provides, or a concurrency restriction or resource constraint prevents the build from starting.

The important idea at this stage is that triggering a build and executing a build are two different events. The Queue is the boundary between those two stages.

Later, when we discuss scaling and troubleshooting, Queue behavior will become one of the most useful signals for understanding whether Jenkins has enough execution capacity or whether the bottleneck exists somewhere else.

6. Pipeline and Jenkinsfile

Modern Jenkins workloads are commonly defined as code using a Jenkinsfile.

To make the execution model concrete, let’s use the Jenkins Pipeline equivalent of “Hello, World!”: a tiny Declarative Pipeline that is easy to read but still contains enough structure to follow a real build from start to finish.

Consider this example:

pipeline { agent any stages { stage('Build') { steps { sh 'echo "building..."' } } stage('Test') { steps { sh 'echo "testing..."' } } stage('Package') { steps { sh 'echo "hello Jenkins" > artifact.txt' archiveArtifacts artifacts: 'artifact.txt' } } } } 

We are not going to explore the entire Declarative Pipeline syntax in this article. Instead, this Jenkinsfile gives us something concrete to trace through the Jenkins execution lifecycle.

The pipeline contains three stages:

The line agent any tells Jenkins that the Pipeline can run on any available execution node. The Controller coordinates the Pipeline, while the execution environment runs the actual shell commands.

That distinction becomes clearer once we walk through what happens after a build is triggered.

7. What Happens After You Click "Build Now"?

With that Jenkinsfile in mind, we can now trace what Jenkins actually does when a build is triggered. This execution lifecycle is the core mental model we will use throughout the rest of the series.

At a high level, the execution path looks like this:

→ Trigger

→ Controller

→ Queue

→ Executor Allocation

→ Workspace

→ Build / Test / Package

→ Artifact + Result

Step 1 — Trigger

A Jenkins build can be triggered manually, by a Git webhook, by a scheduled event, by another pipeline, or through an API call. The trigger simply tells Jenkins that work should be performed, and the Controller receives the request and begins coordinating what happens next.

Step 2 — Queue

Jenkins creates a build request and determines whether suitable execution capacity is available. If an appropriate node and Executor are immediately available, the Queue wait may be almost invisible. If not, the build remains queued until Jenkins can schedule it.

The Queue therefore represents build demand that has not yet been matched with available execution capacity.

Step 3 — Executor Allocation

Once Jenkins identifies a suitable node, it also needs an available Executor on that node. A useful analogy is to think of the Agent as a machine and the Executor as one available seat on that machine.

The new workload can use Executor 2. At this point Jenkins has successfully matched the build request with both an execution environment and an execution slot.

Step 4 — Workspace

Before running commands, Jenkins needs a working directory. That directory is called the Workspace.

A Workspace may contain source code, downloaded dependencies, temporary files, test results, and intermediate build outputs.

Commands such as npm test or mvn package execute from this environment. The key operational point is that a Workspace is a working directory, not permanent storage. It may disappear when a node is rebuilt, the Workspace is cleaned, or an ephemeral execution environment is terminated.

Step 5 — Pipeline Stages Execute

Once the Workspace is prepared, Jenkins begins executing the Pipeline stages. Using our example, the build flows from Build to Test and then Package.

The Controller continues to orchestrate the Pipeline and track its execution state, while the shell commands themselves execute on the selected node. For example, sh 'echo "building..."' is a workload execution step.

A useful way to visualize the responsibility boundary is:

Step 6 — Result and Artifact

When the build finishes, Jenkins records the result along with console logs, timestamps, build numbers, test information, and other metadata.

The Pipeline may also produce files that need to survive after execution, such as JAR files, packages, reports, or deployment bundles. In our example, archiveArtifacts artifacts: 'artifact.txt' tells Jenkins to retain artifact.txt rather than leave it only in the Workspace.

This distinction is important. The Workspace belongs to the execution environment and may be temporary. An Artifact is build output that we intentionally preserve beyond the lifetime of the Workspace or execution environment.

8. JENKINS_HOME: Stateful Controller, HA, and Recovery

So far, we have focused mainly on execution. The next question is where Jenkins keeps its own configuration and operational state.

The answer is primarily JENKINS_HOME. This directory contains important Jenkins data such as job configuration, global configuration, plugin files, build metadata, node configuration, credentials-related configuration, and other state required for the Controller to operate.

The important architectural implication is that the Jenkins Controller is stateful. This directly affects how we design for high availability (HA).

For a typical stateless web application, we can place several identical instances behind a load balancer and replace any failed instance without losing application state. Jenkins does not fit that model so easily.

If multiple Controllers independently modify the same Jenkins state, coordination becomes a serious problem. Simply placing two Jenkins Controllers behind a load balancer and pointing both of them at the same persistent filesystem does not automatically create a safe active-active Jenkins cluster.

Persistent storage solves the durability problem by preserving Jenkins state when the underlying compute is replaced or fails. It does not solve concurrent state modification, Controller coordination, Pipeline execution ownership, plugin state consistency, in-memory execution state, leader election, or safe active-active behavior.

For a traditional Jenkins Controller, HA is often better approached as a durability, recovery, and controlled failover problem rather than an active-active scaling problem. A more realistic design uses one active Controller backed by durable state, with a recovery mechanism capable of starting a replacement Controller when the current compute fails.

That design still requires decisions around health checks, startup behavior, backup, recovery time, and preventing multiple Controllers from incorrectly operating against the same state.

This distinction will become important when we later move the Controller from EC2 to ECS Fargate. The Fargate task running the Controller may be replaceable, but JENKINS_HOME is not. So when we eventually describe Jenkins as running on serverless compute, we need to be precise: the compute layer may be ephemeral, but the Jenkins application remains stateful.

9. Jenkins State, Workspace, and Artifact Are Different Things

Jenkins state, Workspace data, and build Artifacts may all exist during a Pipeline run, but they have very different lifecycles.

Jenkins state must survive Controller replacement and be available during recovery. Workspace data may be safely discarded after execution depending on the workload. Artifacts should survive whenever another build, release process, or deployment needs to consume them later.

A production architecture should therefore avoid treating these storage categories as interchangeable. This becomes especially important once execution environments become ephemeral.

10. What We Have Not Solved Yet

We now understand Jenkins logically, but we have not built anything yet.

There are still several infrastructure questions to answer. Where should the Controller run? Where should JENKINS_HOME live? How should users access Jenkins? How much compute does the Controller need? Should builds execute directly on it? How do we introduce separate Agents, and eventually elastic ones?

Those questions are where AWS enters the story.

The architecture will evolve gradually throughout this series:

We will not jump directly to the final design. Instead, we will build each stage, understand its limitations, and then use those limitations to justify the next architectural step.

Key Takeaways

The Jenkins Controller is the control plane of the system. It coordinates jobs, scheduling, Pipeline execution, configuration, and Jenkins state.

The Queue separates build demand from available execution capacity, so a triggered build does not necessarily start immediately.

An Agent is an execution environment, while an Executor is an execution slot on that environment.

Workspace data, Artifacts, and JENKINS_HOME represent different categories of state with different lifecycles and should be designed accordingly.

Finally, Jenkins is stateful. That statefulness affects not only persistence and backup, but also how we approach High Availability. HA is not simply a matter of running multiple Controllers behind a load balancer; recovery, failover, and state ownership must be designed explicitly.

Next: Installing Jenkins Controller on AWS EC2

We now understand what Jenkins needs logically. The next step is to place those concepts onto real infrastructure.

In the next article, we will provision an Amazon EC2 instance and install a Jenkins Controller from scratch. That will give us our first real Jenkins environment and allow us to run the Pipeline used in this article.

For the first time, we will observe the full execution flow directly in Jenkins: Queue → Executor → Workspace → Console Output → Artifact

That simple EC2 installation will become the baseline architecture that we gradually evolve throughout the rest of the series.