When we talk about High Performance Computing (HPC), we often hear terms such as CPU cores, GPU threads, compute units, parallel processing and heterogeneous computing.
These terms are frequently used when discussing powerful servers, AI systems and scientific applications. But what do they actually mean?
How does a CPU execute instructions? Why are GPUs designed to handle thousands of threads? What makes a CPU different from a GPU? And why do some processors combine CPU and GPU resources within the same architecture?
To understand modern HPC systems, we first need to understand how processors execute work and how their computing resources are organised.
Let's start with the basics.
1. What Does a Processor Actually Do?
At its simplest, a processor executes instructions.
Think about a computer program that needs to calculate the sum of two numbers:
a = 10 b = 20 result = a + b The processor needs to execute instructions to retrieve the values, perform the addition and store the result.
A processor repeatedly performs operations such as fetching instructions, decoding them and executing them.
This is the fundamental job of a CPU or GPU, although their architectures and the way they organise execution are different.
A processor contains several hardware components that work together to execute instructions. These include execution units, registers, control logic and other supporting components.
The important question is how a processor handles multiple operations and tasks.
This brings us to cores and threads.
2. Understanding CPU Cores and Threads
What is a CPU core?
A CPU core is an individual processing unit within a processor.
You can think of a core as a worker who can independently execute instructions.
A CPU with eight physical cores has eight processing units capable of executing different instruction streams concurrently.
For example, imagine an HPC server with an eight core CPU. One core might execute part of a scientific calculation, another might handle a separate process, while other cores work on additional tasks.
This allows multiple workloads to make progress at the same time.
However, having more cores does not automatically make every application faster. The application must be designed to divide its work across multiple cores.
What is a thread?
A thread is an execution path within a program. It represents a sequence of instructions that can make progress independently of other threads.
Imagine processing a dataset containing one million records.
Instead of processing every record sequentially, an application could divide the dataset into smaller sections and assign each section to a different thread.
Each thread works on its assigned section. If the system has enough available processing resources, multiple threads can execute concurrently.
There is also an important distinction between software threads and hardware threads.
A software thread is created by an application or runtime. A hardware thread is an execution context supported by the CPU.
Modern CPUs can use Simultaneous Multithreading (SMT), allowing one physical core to maintain multiple hardware execution contexts.
For example, an eight core CPU with two way SMT may expose 16 logical CPUs to the operating system.
This does not mean the processor has 16 physical cores. The two hardware threads on each core share some of the same underlying execution resources.
How does this relate to HPC?
In HPC environments, applications frequently use technologies such as OpenMP and MPI.
OpenMP allows applications to create multiple threads that can execute across CPU cores within a shared memory system.
MPI allows separate processes to communicate, including across different compute nodes.
For example, a scientific application might use MPI to distribute work across four compute nodes and OpenMP to create multiple threads within each node.
This combination is commonly used in parallel HPC applications.
3. Understanding GPUs and GPU Threads
A GPU, or Graphics Processing Unit, is designed to handle large amounts of parallel computation.
Although GPUs were originally developed primarily for graphics processing, their ability to perform many similar calculations simultaneously made them extremely useful for scientific computing, simulations and AI.
A CPU is generally designed to handle a wide variety of tasks, including complex instructions and different execution paths.
A GPU, on the other hand, is designed to execute large numbers of operations in parallel.
Consider an application that needs to add two arrays containing one million numbers.
The calculation for each element is independent of the others.
A GPU can divide this work into thousands of smaller execution tasks, allowing many calculations to be performed concurrently.
What is a GPU thread?
A GPU thread represents an individual execution instance that performs instructions on its assigned data.
For example, one GPU thread might calculate the result for the first element of an array, while another calculates the second element.
Thousands of these threads can be organised and scheduled by the GPU.
In NVIDIA CUDA terminology, threads are grouped into blocks, and blocks are organised into a grid.
Threads are also executed in groups called warps. On NVIDIA GPUs, a warp typically contains 32 threads.
The hardware schedules these warps across its Streaming Multiprocessors (SMs).
It is important to understand that a GPU thread is not the same as a physical CPU core. Thousands of GPU threads do not require thousands of separate physical processing units.
Instead, the GPU hardware schedules and executes groups of threads across its available execution resources.
This is one of the reasons GPUs can support such a large number of parallel tasks.
CPU threads vs GPU threads
Although both CPUs and GPUs use threads, they organise and execute them differently.
CPU threads are generally designed to handle varied instruction sequences and more independent execution paths.
GPU threads are designed to support large scale parallelism, often performing similar operations across large collections of data.
For example, a CPU might handle the application logic, prepare data and coordinate the workload, while a GPU performs thousands of mathematical operations on that data.
Both are important, but they are optimised for different types of work.
4. Why Do Different Processors Have Different Architectures?
Now that we understand CPU cores and GPU threads, we can look at how processors are designed.
A compute architecture describes how processing resources are organised and how they interact with other components of a system.
One important difference between processor architectures is how many types of processing resources are available and how they access memory.
Let's look at three common examples.
5. CPU, Discrete GPU and Integrated GPU Architectures
CPU based architecture
In a traditional CPU based system, the CPU performs the main processing operations.
It contains multiple cores, and each core can execute instructions independently.
The CPU also has its own memory controllers and cache hierarchy to support efficient data access.
This architecture is common in traditional servers and many HPC compute nodes.
Discrete GPU architecture
A discrete GPU is a separate processor installed in a system, usually connected through PCIe or another high speed interconnect.
It has its own processing resources and dedicated memory.
Imagine an HPC server containing a CPU and a separate GPU.
The CPU handles general application tasks, while the GPU handles workloads that benefit from massive parallel computation.
In this architecture, the CPU and GPU generally operate with separate physical memory spaces.
This means the application or its runtime needs to manage how data becomes available to each processor.
The separation between these memory spaces is an important architectural consideration when designing GPU accelerated applications.
Integrated GPU and APU architecture
An integrated GPU is built into the same processor package or system on chip as the CPU.
Many integrated GPU designs share system memory with the CPU.
An APU, or Accelerated Processing Unit, is a term commonly associated with AMD processors that combine CPU and GPU capabilities within one chip or package.
In these architectures, CPU and GPU compute resources can access a shared memory space, depending on the hardware design.
This can reduce the need to transfer data between completely separate physical memory spaces.
However, shared memory does not mean that CPU and GPU resources have identical performance characteristics. They may compete for memory bandwidth and other shared resources.
The important distinction is how the processing units and memory are organised, rather than simply whether the CPU and GPU are located on the same chip.
6. What Is Heterogeneous Computing?
Modern HPC and AI systems increasingly combine different types of processing resources.
This approach is called heterogeneous computing.
Instead of relying on only one type of processor, a system combines resources such as CPUs and GPUs so that each can perform the work it is best suited for.
For example, an AI training workload might use:
- CPU cores to manage the application and prepare data.
- GPU compute units to perform mathematical operations.
- Multiple GPUs to distribute larger workloads.
- High speed networking to exchange information between GPUs across different servers.
This combination allows applications to take advantage of different processor architectures.
It is also important to understand that heterogeneous computing is not limited to CPUs and GPUs. Modern systems can include other specialised processing resources such as AI accelerators and data processing units.
The objective is to improve overall system performance by selecting the right processing resources for different tasks.
7. Why Does Memory Architecture Matter?
Although this article focuses on processors, memory is an important part of understanding compute architecture.
A processor needs access to instructions and data before it can perform calculations.
However, different processor designs organise memory differently.
For example, a traditional CPU system uses system memory accessed through the CPU memory controllers.
A discrete GPU generally has its own dedicated memory, while many integrated GPU designs share system memory with the CPU.
These differences affect how applications are designed and how data moves through a system.
They also influence performance, particularly when a workload needs to move large amounts of data between processing resources.
Memory hierarchy, cache, bandwidth and High Bandwidth Memory (HBM) are important topics in their own right. We will explore these in more detail in a separate article.
8. What Comes Next in Processor Design?
Modern processor designs are becoming increasingly complex.
Traditional processors were often designed around a single large silicon die containing the main processing components.
However, manufacturing increasingly large and complex chips introduces challenges related to manufacturing yields, physical size, cost and power.
This is where chiplet architecture becomes important.
Instead of building everything into one large die, manufacturers can combine multiple smaller dies within a single package.
Different chiplets can perform different functions, such as CPU processing, GPU computation and I/O operations.
Advanced packaging technologies allow these components to communicate with one another at high speeds.
Some modern designs also combine compute chiplets with High Bandwidth Memory (HBM), bringing large memory bandwidth close to the processing resources.
These technologies are helping shape the next generation of HPC and AI processors.
We will explore chiplets, HBM and advanced processor packaging in upcoming articles.
Conclusion
Understanding modern processors starts with understanding how they execute work.
CPU cores provide processing resources for executing instruction streams, while threads allow applications to divide work into smaller execution paths.
GPUs take parallelism further by organising large numbers of threads across their available execution resources.
Different processor architectures also organise compute resources and memory in different ways. This includes traditional CPU systems, discrete GPUs and integrated CPU and GPU designs.
These fundamentals are important for anyone learning HPC because they help explain why some applications benefit from more CPU cores, why others perform better on GPUs, and why modern systems increasingly combine different processing architectures.
Before exploring advanced concepts such as chiplets, HBM and heterogeneous integration, it is important to understand how the processors themselves work.
That is the foundation of modern high performance computing.
