Loading lesson…
Computing Foundations for AI / Hardware Basics
The worker, the workbench, and the warehouse.
Reviewed by Yuvaraj
Open any program and three pieces of hardware immediately start working together: the CPU, the GPU, and RAM. It is tempting to lump them together as "the fast parts of the computer," but each was built for a different job. Understanding how they differ is the first real step toward understanding why running and training AI models demands the specific hardware it does.
The central processing unit (CPU) is the general-purpose brain of the machine. It runs your operating system, your applications, and the logic that decides what happens next. A modern CPU has roughly 8 to 64 cores, and each core is genuinely powerful: it runs at billions of cycles per second, predicts which branch of an if statement will be taken, and reorders instructions so it rarely sits idle. This design optimizes for latency, finishing one task, especially a complicated and unpredictable one, as quickly as possible.
The graphics processing unit (GPU) takes the opposite approach. Instead of a few clever cores, it packs thousands of small arithmetic units that all run the same instruction at the same time on different data (a model often called SIMD or SIMT). Each unit is simpler and slower than a CPU core and is poor at branchy logic, but there are so many of them that the GPU performs enormous amounts of arithmetic in parallel. This design optimizes for throughput, total work completed per second, not the speed of any single step. That is exactly why GPUs power deep learning, which is mostly large, repetitive matrix math.
RAM (random-access memory) is the fast working memory the processor reads from and writes to while a program runs. It is measured in gigabytes (commonly 8 GB to 128 GB) and delivers data at a bandwidth of tens to hundreds of GB/s. Crucially, RAM is : when the power turns off, its contents vanish. That makes it completely different from (an SSD or hard disk), which is slower but keeps your files permanently.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
The memory hierarchy in one line
Registers and CPU cache are faster and smaller than RAM; RAM is faster and smaller than an SSD or hard disk. Data is constantly moved up this ladder to be worked on and back down to be saved.
Suppose we must compute a multiply-add, , one million () times, and every one of the million calculations is independent, using its own separate inputs. No calculation needs another's result.
A GPU shines here. It hands thousands of its cores thousands of the multiply-adds at once, chews through all million values in a small number of parallel passes, and finishes far sooner. A CPU with, say, 16 cores can only work on a few dozen at a time, so it must grind through many more sequential rounds.
Now flip the task: a long chain of dependent, branchy steps, where step 2 needs step 1's answer, step 3 branches on step 2's, and so on. Here parallelism is useless, there is nothing to run side by side. The GPU's thousands of cores sit idle waiting on each other, and its weak branch handling hurts. The CPU's fast cores, branch prediction, and out-of-order execution win easily.
| Component | Role | Strength | Typical size |
|---|---|---|---|
| CPU | General-purpose brain; runs the OS and program logic | Low latency on branchy, sequential work | ~8–64 cores |
| GPU | Massively parallel math engine | High throughput on uniform, parallel work | Thousands of arithmetic units |
| RAM | Volatile working memory the processor reads and writes | Fast access to active data | ~8–128 GB, tens–hundreds of GB/s |
Common mistakes