Cores & Threads

Introduction

For decades, CPU performance improved primarily through higher clock speeds. But around 2005, chip manufacturers hit a wall — the "frequency wall" — where increasing clock speed produced too much heat and power consumption. The industry needed a new approach, and the answer was multi-core processors.

Instead of making one core faster, manufacturers began putting two, four, eight, or even more cores on a single chip. Each core is essentially a complete CPU — with its own ALU, control unit, and cache — capable of running its own program independently. This meant that a dual-core processor could theoretically do twice as much work as a single-core processor at the same clock speed, as long as the software was written to take advantage of multiple cores.

Alongside multiple cores came threading — the ability to split a program into multiple streams of execution that can run concurrently. Technologies like Intel's Hyper-Threading (and AMD's SMT) allow a single core to handle two threads simultaneously, further improving efficiency by keeping the core's execution units busy even when one thread stalls.

How It Works

Deeper Dive

A physical core is a complete, independent processing unit on the CPU die with its own dedicated execution resources. A logical core (or thread) represents a virtual processing unit created by technologies like Hyper-Threading or SMT. With Hyper-Threading, a single physical core can appear as two logical cores to the operating system, allowing it to run two threads simultaneously.

The operating system's scheduler assigns threads to logical cores. If a physical core supports two threads, and one thread stalls (waiting for memory, for example), the other thread can use the core's execution units that would otherwise be idle. This typically provides a 15-30% performance improvement over a single-threaded core, though the exact gain depends on the workload.

Multi-Core Architectures

Modern CPUs use different multi-core architectures depending on their target market. Desktop processors like the AMD Ryzen and Intel Core series typically feature 6 to 16 high-performance cores. Server processors like AMD EPYC or Intel Xeon can have 64 to 128 cores or more. Mobile processors like Apple's M-series use a hybrid architecture — a mix of high-performance cores (for demanding tasks) and high-efficiency cores (for background tasks), optimising battery life while maintaining responsiveness.

The cores on a multi-core CPU share certain resources, particularly the last-level cache (L3 cache) and the memory controller. This shared cache is beneficial because if one core accesses data from memory, it becomes available to all other cores at cache speed. However, sharing also creates challenges — if multiple cores try to write to the same memory address simultaneously, the cache coherence protocol must ensure all cores see a consistent view of memory.

Parallelism: Task vs. Data

Multi-core processors enable two main types of parallelism. Task parallelism divides a program into different tasks that can run concurrently — for example, a web browser might have one thread rendering a page, another downloading an image, and a third handling user input. Data parallelism splits a large dataset across multiple cores so that each core works on a subset — for example, processing 1000 images on 4 cores, each handling 250 images.

Not all programs benefit equally from multiple cores. Workloads that are embarrassingly parallel (like video rendering, scientific simulations, or machine learning training) can scale almost linearly with core count. Sequential workloads (like most legacy games or single-threaded applications) see little benefit beyond one or two cores. This is why single-core performance still matters, and why CPUs with fewer but faster cores can outperform chips with many slow cores on certain tasks.

Cache Coherency and Coherence Protocols

When multiple cores each have their own L1 and L2 caches, a problem arises: if core A modifies a value in its cache, core B might still have the old value in its cache. The cache coherence protocol (such as MESI or MOESI) ensures that all cores see a consistent view of memory. When a core writes to a cached address, it broadcasts a message to other cores, invalidating their cached copies so they must fetch the updated version.

This coherence traffic is a necessary overhead of multi-core systems. As core counts increase, the amount of coherence traffic grows, potentially becoming a bottleneck. This is known as the coherence wall, and it is one reason why scaling to hundreds of cores is significantly more challenging than just putting more cores on a die.

Advanced

At a deeper level, cores & threads involves rules and patterns that engineers use worldwide. Core follows standards so different brands and devices can still work together. That is why your phone, school laptop, and game console can all connect to the same network or use the same apps.

Thread does not happen in a straight line. Systems often use backup paths, error checking, and retries so information arrives correctly. When something fails, smart Hyper-Threading design helps the system recover instead of shutting down completely.

Scientists and engineers keep improving these systems every year — making them faster, safer, and more energy-efficient. The ideas you learn in this chapter are the same building blocks used in real data centers, robots, apps, and websites around the world.

Vocabulary Table

Fun Facts

Interactive Diagram

Launch the interactive diagram to see Cores & Threads in action.

Open Interactive Diagram

---

The interactive diagram for this chapter demonstrates Cores and Threads. It shows a multi-core CPU with threads being distributed across cores for parallel processing.

Knowledge Check

Chapter Navigation