There’s a very common instinct in backend system design:

if we want performance, keep everything in memory

And the corollary:

disk is slow, avoid it as much as possible

It feels obvious. Memory is fast, disk is slow, so we optimize around memory. This mental model is so common that most systems are designed around it. Keep hot data in RAM, build caches, flush to disk only when necessary, and hope we don’t hit IO too often.

Then we look at something like Apache Kafka, and it seems to violate that rule completely. Kafka writes everything to disk immediately. It doesn’t try to keep messages in heap. It doesn’t rely on any fancy in-memory structures.

The key to understanding this is not that disks are magically fast. It’s that they are predictably fast under the right access pattern, and modern operating systems are extremely good at hiding their weaknesses. When done right, In many cases, a system designed around the filesystem rather than around in-memory data structures can match or even outperform what we would expect from a “memory-first” design.

Disk performance is not uniform

When people say disk is slow, they’re usually thinking about latency. And they’re not wrong. If we issue a random read or write, the disk head in older hard disks had to move to a location on the disk, and that takes several milliseconds, which is indeed very slow compared to RAM access.

But disks don’t just have latency. They also have throughput. And throughput behaves very differently depending on access patterns.

If we issue random reads and writes across the disk, every operation requires a seek because the disk head has to constantly move around. That means we are constantly paying that latency cost. This is exactly what many traditional data structures do. Trees, indexes, and in-place updates all lead to scattered access patterns, which in-turn mean repeated seeks.

If we access disk sequentially by just reading or writing continuously next to each other, one block after another, the disk head barely moves. There are no repeated seeks, and the disk can stream data at a very high rate at hundreds of megabytes per second on commodity hardware.

The performance gap between these two modes is enormous. Thousands of times difference on the same hardware.

Most traditional data structures like B-trees, indexes, anything involving updates in place, force random access patterns in disk through pointers. This means the disk head has to jump between addresses pointed by the pointer in the disk. That’s why they perform poorly on disk.

Kafka avoids this entirely by doing one thing: append-only writes to files. Just write to the end of a log file. These data are written next to each other block after block. And hence the disk head doesn’t have to move a big distance to get the next data block.

This is one of the design decisions that gives Kafka the fastest performance hard disks can offer. An another thing is caching.

The operating system is already caching everything

On systems like Linux, disk IO doesn’t go straight to the physical disk. It goes through the page cache, which lives in RAM.

Most modern operating systems maintain a unified page cache. Any file we read or write goes through this cache. When our application writes to a file, the data is typically placed into memory first. The OS then decides when to flush it to disk, grouping small writes into larger, more efficient operations. This is often referred to as write-behind.

When Kafka writes data:

• the data is copied into the page cache
• the OS acknowledges the write
• the actual disk flush happens later, in large batches

So from Kafka’s perspective, writing to disk is actually writing to memory most of the time.

When doing reads as well, the OS performs read-ahead. That is, If it detects sequential access i.e. the subsequent data block will be requested next, it will prefetch additional data into memory before our application even requests it. As a result, many reads are served directly from RAM, even though our application is technically reading from the disk.

What this means in practice is that a filesystem-based design is not giving up memory performance. It is outsourcing memory management to the OS, which is highly optimized for this exact purpose.

But is it worth building our own cache system?

At this point, it’s tempting to say: but why not skip disk entirely and just build a pure in-memory system by building a caching layer in Apache Kafka itself at the application level?

There are two major problems with that, especially in JVM-based environments.

The first is memory efficiency.

When we store data as objects, we’re not just storing the data itself. We’re storing metadata, object headers, alignment padding, and references to other objects. A small logical record can easily take up 2–5x more space in memory than its raw representation.

Kafka doesn’t store messages as objects. It stores them as compact byte arrays on disk. That means far better density and far more efficient use of memory when those bytes are cached.

The second problem is garbage collection.

The second issue is garbage collection. As the amount of in-heap data grows, the cost of managing that memory increases. Garbage collectors need to traverse object graphs, track references, and sometimes move objects around. Large heaps often lead to unpredictable latency and long pauses.

If we try to keep tens of gigabytes of active data in memory, we will notice it. GC becomes one of our biggest bottlenecks.

Kafka sidesteps all of this by keeping most data out of the heap. When we store data as raw bytes on disk instead, we avoid both issues. The data is compact, and it is not subject to garbage collection. Memory is still used via the page cache but it is managed outside the application’s heap by the OS itself, eliminating GC overhead.

This leads to a counterintuitive outcome.

If we build our own in-memory cache in our application, we are constrained by:

• heap size
• object overhead
• GC limits

We often end up using less memory than the machine actually has, just to keep GC under control.

If we rely on the filesystem and page cache, the OS will happily use almost all available memory for caching file data. And because the data is stored compactly, we get far better utilization. On a machine with 32GB of RAM, it’s entirely possible to have 28–30GB effectively acting as cache, without touching our application heap.

So instead of losing performance by going to disk, we actually gain effective memory capacity.

Warm cache

Another advantage of this approach is when we consider restarts.

In-memory systems lose their cache when the process dies. The system must rebuild it, which can take time and degrade performance during that period. In some cases, this warm-up phase can be significant enough to impact availability.

With a filesystem-based approach, the data is already on disk. More importantly, much of it may still be in the OS page cache in the RAM even after the process restarts.

The bigger takeaway

The important lesson here is not that disk is fast or that memory is unnecessary. It’s that performance depends heavily on access patterns and system design.

A system that aligns with the hardware by using sequential IO and leveraging the OS page cache can achieve high throughput, predictable latency, and simpler architecture. In that sense, the filesystem is not a bottleneck to be avoided. It is a tool that, when used correctly, let’s systems scale in a way that purely in-memory designs often cannot.