Object Storage Enters the Training Loop: S3-over-RDMA in 2026

Overview

For most of the last decade the answer to “where does AI training data live” was a parallel filesystem, with object storage sitting behind it as the cheap archive. That separation is dissolving. In 2026, vendors are shipping S3 implementations that run over RDMA and, in some cases, execute inside a DPU with no host CPU in the path at all. The practical consequence for anyone designing an on-prem AI cluster is that the object tier is no longer just capacity — it can be a performance tier, and the number of distinct storage systems you have to buy and operate may fall.

Why object storage sat outside the training loop

Object storage won the capacity argument years ago: flat namespace, erasure coding, commodity nodes, no metadata server to melt down. What it lost was the latency argument. Classic S3 is HTTP over TCP, which means every request pays for connection handling, header parsing and multiple memory copies through the host network stack. At the aggregate throughput a GPU cluster demands, that overhead consumes CPU cores that were bought to feed GPUs, and the tail latency it introduces shows up as GPU idle time.

So the standard architecture became two systems: a high-performance parallel filesystem for the active working set, and object storage behind it for everything else, with staging jobs copying data between them. That works, but it costs a second storage platform, a second operations skill set, and a permanent data-management chore. The pattern is described in more detail in our note on storage architecture for AI training.

What S3-over-RDMA changes

The recent work maps the S3 object protocol onto RDMA rather than TCP. RDMA lets one machine read or write another machine’s memory without involving the remote CPU or traversing the kernel network stack, which removes exactly the two costs that kept object storage out of the hot path. MinIO reports that its S3-over-RDMA path reduces latency and host CPU consumption, with the largest benefit on large objects and sustained throughput — which is precisely the access pattern of training data loading and checkpoint writes.

The important nuance is that this does not make object storage good at small random I/O. Object stores remain optimised for large sequential objects. What changes is that the large-object case, previously handicapped by protocol overhead, now runs close to wire speed. If your training data is sharded into large files (WebDataset-style shards, packed token files, large image tars), that is the case you actually have.

DPU-resident object stores

The second development pushes further: running the object storage software inside the DPU itself. NVIDIA’s BlueField-4 combines a Vera CPU with a ConnectX-9 class network interface and is positioned as a storage platform, handling I/O scheduling, protocol conversion, compression and encryption without host CPU involvement, and supporting NVMe-oF, GPUDirect Storage and the DOCA acceleration libraries. Vendors including MinIO and WEKA have announced object and filesystem software that runs natively on it, with general availability reported for the second half of 2026.

Architecturally this turns a JBOF into a self-contained storage node: NVMe on one side, 800 Gb/s network on the other, and the storage stack executing on the DPU in between. There is no storage server to size, patch or fail. For a mid-size Indian deployment that has struggled to justify a full parallel-filesystem team, that reduction in operational surface may matter more than the throughput number.

Where each tier now belongs

The direction of travel is consolidation: one namespace, several media classes, fewer copies. The tier that has genuinely grown rather than merged is the inference-side cache described in KV cache offloading, which is a different workload with different latency requirements.

How to decide for a 2026 build

Start with the access pattern, not the product. Profile object size and read pattern in your actual pipeline. If the working set is large sequential shards, an RDMA-capable object tier is a credible primary. If you have millions of small files, metadata-heavy access or POSIX-dependent tooling, a parallel filesystem still earns its place and the object tier stays behind it.

Second, be conservative about timelines. DPU-resident storage products reach general availability through 2026, and first-generation platform software deserves a pilot rather than a production bet. Design the reference architecture so the object tier can be served conventionally at first and moved to an RDMA or DPU path later without redesigning the namespace. Third, size capacity against the checkpoint and dataset arithmetic in GPU storage planning for checkpoints and RAG indexes before choosing a protocol — capacity mistakes are more expensive than protocol mistakes.

The India angle

Two local factors push toward this architecture. First, operational staffing: parallel filesystems are skill-intensive, and teams able to run one at scale are scarce and expensive in India. An object namespace that most engineers already understand, with the performance stack pushed into the DPU, lowers that barrier materially. Second, data governance: a single in-country object namespace with clear bucket-level policy is far easier to evidence under DPDP than a two-system architecture with staging copies scattered between them. For regulated buyers, the audit story is part of the design, as covered in AI factory storage planning.

Frequently asked questions