In-Memory Computing

Explainer

In-Memory Computing explained

In-memory computing does calculations inside or right next to the memory that stores the data, instead of copying the data to a separate processor. It is also called processing-in-memory (PIM) or compute-in-memory.

In a conventional computer, the processor and the memory are separate, and every calculation needs data to travel between them. This is the von Neumann bottleneck. In-memory designs reduce that travel, either by adding small compute units to memory chips or by using the memory cells themselves to perform operations such as multiply-and-add.

How it differs from the usual approach

In a normal processor, moving a number from external memory costs far more energy than doing arithmetic on it, by orders of magnitude. In-memory computing attacks that cost directly. Digital versions put logic inside or beside memory arrays and stay exact. Analog versions use the physics of the memory cells and are efficient but less precise. Software must be adapted to use either, which has limited adoption.

Where it is used

• Neural network inference, where the same weights are read again and again.
• Database scans and search that read huge amounts of data.
• Genomics and graph analytics that are limited by memory access.
• Low-power AI in edge devices.
• Research on memory-bound parts of large language models.

What to watch

• Memory makers have shown DRAM with built-in processing units, including SK hynix and Samsung prototypes and products.
• Several startups build in-memory AI chips using SRAM, flash or resistive memory.
• Programming models and software support are the main barrier to broad use.
• Memory-bound AI inference keeps interest in the field high, because it often limits speed more than arithmetic does.

Frequently asked questions