Skip to main content

Technology

GPUs

Graphics Processing Units enabling parallel computing for AI, gaming, scientific simulation, and cryptocurrency mining.

19 companies

Explainer

GPUs explained

A GPU, or graphics processing unit, is a chip with thousands of small cores that can run the same operation on a lot of data at the same time. It was first built to draw images in games. The same design suits any job that applies simple maths to large arrays of numbers.

Training and running neural networks is mostly matrix multiplication, which fits this design well. Modern data center GPUs add dedicated matrix units (NVIDIA calls them Tensor Cores), fast stacked HBM memory and high-speed links to other GPUs. Software layers such as CUDA from NVIDIA and ROCm from AMD let programmers use them.

How it differs from the usual approach

A CPU is built to finish one varied task quickly. A GPU is built to finish a huge number of similar tasks at once, which is called throughput. That makes GPUs far faster than CPUs on parallel work such as graphics and neural networks, and less suited to code with many branches. Compared with fixed-function AI chips, GPUs stay programmable, and that flexibility is a large part of their market position. So is their software ecosystem, which took years to build.

Where it is used

  • Training large AI models in data centers.
  • Running AI inference for chatbots, search and image generation.
  • Rendering graphics in games, film and design software.
  • Scientific simulation on supercomputers.
  • Video encoding and analytics.

What to watch

  • NVIDIA introduced CUDA in 2007 and still leads AI training. AMD and Intel are the other large GPU makers, and cloud providers also build their own accelerators.
  • The US has restricted exports of the most advanced AI chips to some countries since October 2022, which shapes where and how they are sold.
  • Supply of HBM memory and advanced packaging is a recurring constraint on how many GPUs can be built.
  • Software portability layers aim to let models run on non-NVIDIA hardware without rewriting code.

Frequently asked questions

Why are GPUs used for AI?

Neural networks are mostly large matrix multiplications, and GPUs do many such operations in parallel. They also have fast memory and mature software, so they are practical to use.

What is the difference between a GPU and a TPU?

A GPU is a programmable parallel processor that began in graphics. A TPU is Google's chip designed specifically for the matrix maths in machine learning. TPUs are less flexible but can be efficient on the workloads they target.

Why is NVIDIA so dominant?

Its hardware is strong, and it also built the CUDA software ecosystem over many years. Most AI code and tools are written for it, which makes switching costly.