CODE IN TOKYO

From an Engineer's Perspective: What Is the Difference Between a CPU and a GPU?

The Word GPU Keeps Coming Up in AI

Recently, whenever people talk about AI, the word “GPU” almost always comes up.

I vaguely understood that AI needs GPUs and that GPUs are faster than CPUs. But when I tried to explain the essential difference, I realized it was surprisingly difficult.

So I looked into the circuit-level differences and why their roles are so different.

The CPU Is a Particular Pastry Chef, the GPU Is a Factory Team

The easiest way to explain the difference may be this: a CPU is like a pastry chef who can make one perfect custom cake, while a GPU is like a team of workers on a factory line.

CPU: The Skilled Pastry Chef

A CPU is like a brilliant pastry chef.

When a customer says:

the chef can respond flexibly on the spot.

They change the ingredients, adjust the process, and sometimes reorganize the entire workflow.

A CPU is good at looking at the situation and deciding what to do next.

At the circuit level, much of a CPU is made up of control logic and cache memory for deciding what should happen next.

It is excellent at carefully completing one task. But it is not good at making thousands of cakes at the same time.

GPU: The Factory Line Team

A GPU, on the other hand, is like a team of thousands of factory workers.

They are not as smart as the CPU.

If you say:

they get stuck very quickly.

But if you say:

then they become incredibly fast.

Because thousands of workers perform the same operation at once, the processing power is enormous.

The circuit structure is also the opposite of a CPU. A GPU reduces complex control logic and uses that space for a large number of arithmetic units, or ALUs.

In short, it is not intelligence. It is muscle.

Why AI Needs GPUs

At this point, a natural question comes up.

If the pastry chef, or CPU, is more capable, why not do everything on the CPU?

The reason is that most AI training and inference is made up of huge matrix multiplications.

In other words, AI repeats simple operations such as:

It repeats that hundreds of millions or even trillions of times.

That is exactly factory work.

The CPU proceeds while thinking:

The GPU’s style is:

AI needed the latter.

That is why GPUs became the main player.

GPUs Are Not Universal

What I learned is that GPUs are definitely not universal.

To use the same analogy:

The CPU is a made-to-order specialty shop.

The GPU is a massive production factory.

It is not about which one is better. They are simply suited to different jobs.

Conclusion

What would happen in a world with only GPUs?

“Please add more strawberries.”

“Cannot do that.”

“Please make it a chocolate cake.”

“Cannot do that.”

“It is a birthday, so please add a message.”

“Cannot do that.”

But the moment you say:

“Make 10,000 identical strawberry cakes.”

it suddenly becomes unbelievably motivated.

That is a GPU.

When we look at AI, we often feel that it is smart or that it thinks like a human.

But behind the scenes, what is happening is actually thousands or tens of thousands of arithmetic units repeating the same calculations over and over.

When I think that the true identity of an AI that speaks like a philosopher or novelist is actually a terrifyingly muscular calculation team, I find it a little funny.

Maybe cutting-edge technology is often like that.

Bonus: I Also Looked Into TPUs

While reading about GPUs, I became curious about one more thing.

It was Google’s TPU, or Tensor Processing Unit.

It seems to be an even more AI-specialized semiconductor than a GPU.

This is where the difference from GPUs becomes important.

GPUs were originally semiconductors for rendering 3D graphics in games at high speed. They were good at applying similar calculations to huge numbers of pixels and polygons at once.

That property happened to fit well with AI matrix calculations.

In other words, a GPU is like a factory originally built for games that was repurposed into a huge AI calculation factory.

It is not as flexible as a CPU, but a GPU is still a team of humans.

There is still room to change the recipe.

A TPU, on the other hand, is designed from the beginning to focus on AI matrix calculations.

Using the same cake factory analogy, a TPU is like a group of ultra-fast robots lined up inside the factory, specialized only for spreading cream on sponge cakes.

A GPU factory still has some versatility.

It can knead dough, spread cream, and handle packaging. It can also deal with some recipe changes.

But that versatility also creates waste.

A TPU makes a much stronger trade-off.

It focuses on matrix operations, the core calculation used heavily in AI, and strips away unnecessary circuitry.

Instead of taking cakes off the conveyor belt one by one, it spreads the cream while they are still moving and passes them straight to the next step.

TPUs use a structure called a systolic array, where data flows regularly through arithmetic units and calculations are passed along from one unit to the next.

With this structure, TPUs can be even more efficient than GPUs for fixed, large-scale workloads such as matrix calculations.

But they also have the weakness of being too specialized.

A TPU is a robot, so it is not good at complex tasks outside the recipe.

It does not make situational judgments like a CPU, and it cannot be repurposed for similar workloads as broadly as a GPU.

It is fully committed to the matrix calculations at the center of AI training and inference.

So CPU and GPU handle non-AI processing and complex control, while the TPU focuses only on the calculations it is good at.

If a GPU is a factory originally built for games and repurposed for AI, then a TPU is probably best described as a specialized factory designed from the start only for AI matrix calculations.