We just received 2.9M in seed funding 🎉 Read about it here.

Watch our kMetal webinar on Replay here.

Industry Insights

Who Owns the Switch? Network Offloading for Cloud Infrastructure

Who controls the switch tenant traffic passes through decides your CPU bill. A look at network isolation, overlay costs, and moving the work onto DPUs.

Photo by Taylor Vick on Unsplash
Written by
Published on
October 19, 2026
Read Time
10 min

Share

Most digital infrastructure is shared. Cloud providers, the GPU clouds that rent capacity for running AI models, and enterprises that run internal platforms for their own teams all place many tenants on the same servers. Sharing keeps costs down. A server used by several tenants is busy more of the time than one reserved for a single customer, and tenants can grow or shrink without buying hardware.

AI is moving more infrastructure in this direction. As companies put AI to work in their own business, many teams and applications need to run models for a wide variety of tasks. Each needs a slice of GPU capacity rather than a whole cluster, and that needs changes with the project and the time of day. Building a separate, fully isolated environment for each of them is rarely practical, so AI infrastructure is increasingly shared too.

Sharing only works if tenants are fully isolated. Their network traffic should not reach anyone else's, and nobody should be able to join a network they were not given. Most platforms handle this in software through overlay on the server itself, and that works well. It has a cost, though, and the cost is CPU.

That raises a question: who controls the switch that tenant traffic passes through? The answer decides how much CPU goes to networking and who can change the network rules.

This article covers where network isolation is enforced, what each approach costs, and why more platforms are moving that work onto a processor on the network card, called Data Processing Unit (DPU).

This focuses on the network infrastructure that enables connectivity between tenants and the outside world, rather than the GPU-to-GPU interconnects within AI clusters, and concludes with an overview of kMetal, the platform we develop and operate.

Three ways to keep tenants apart

There have been roughly three generations of network isolation in shared data centres.

VLANs. The first approach splits the physical network into virtual segments, configured on the switch ports. A server plugged into a port belongs to that port's segment. This is simple and needs no work on the server, but it does not scale well. The number of segments is limited, and every change means reconfiguring switches. Isolation is tied to cabling and port assignments, which makes it slow to change.

Overlays. The second approach builds each tenant's network in software on top of a shared physical network. A software switch on each server wraps every packet leaving a tenant's virtual machine in an extra header that marks its tenant network, and the receiving server unwraps it. This wrapping is called encapsulation. The physical network only sees traffic between servers. Most cloud and Kubernetes platforms work this way today, kMetal included.

Overlays solved the scaling problem, and when the operator controls the server and a hypervisor separates the tenants, the isolation is sound. The cost is extra work for the server's CPU. Every packet is wrapped and unwrapped, connections are tracked so firewall rules can be applied, and addresses are translated when traffic leaves the tenant network. All of this runs on the same cores the server is meant to sell to tenants.

DPU enforcement. The third approach keeps the overlay but moves the software switch off the server's CPU and onto the network card. A DPU is a network card with its own processor, memory and operating system. The tenant networks are built and enforced there, and the server sees an ordinary network interface with SR-IOV capabilities.

Table – Approach, Where the Isolation Lives, and Main Weaknesses
Table – Approach, Where the Isolation Lives, and Main Weaknesses

What software networking costs

The CPU cost of an overlay rarely shows up on a bill or a dashboard, but it affects a platform in three ways.

The first is capacity. Cores spent processing network traffic cannot be given to tenants. On a platform that sells virtual CPUs, that is lost capacity on every server, and it grows with how much traffic the tenants send to each other.

The second is predictability. When network processing competes with tenant workloads for the same cores, response times become less even. For most applications the effect is small. For interactive services such as AI inference, where people are waiting for an answer, uneven response times are noticed directly.

The third is a ceiling. At high link speeds, software networking can run out of CPU before it runs out of bandwidth. Red Hat tested two servers with 100 Gbit links, connected directly, running an overlay with encryption. With the encryption running on the servers' own CPUs, they could not reach 100 Gbit, because the CPUs were saturated first.

Overlays are still a sound design. The work has to run somewhere, and on a platform that sells CPU time, the server's CPU is a costly place to run it.

The pattern is not new

Large cloud providers made this move years ago. AWS built its Nitro system so that networking, storage and management run on dedicated cards, leaving the server's CPUs entirely to customers. Microsoft described a similar approach for Azure in 2018, moving the work of its software network onto programmable network cards because the host cores were worth more running customer workloads.

The difference today is that operators no longer need to build their own hardware for this. DPUs such as NVIDIA's BlueField can be bought off the shelf, so the same design is possible outside the largest clouds.

How a DPU handles tenant traffic

While implementation details differ across vendors, the following discussion focuses on the NVIDIA BlueField architecture, which serves as the basis for our current integration.

A BlueField card contains a set of Arm processor cores and a hardware switch.The card can run in one of two ways: NIC Mode or DPU Mode. Either the server controls it and it behaves like a plain network card in NIC Mode, or the DPU controls it and the server's software has no say over the network in DPU Mode. There is no supported middle ground where the server keeps its software switch and hands part of the work to the card.

When the DPU is in control, a tenant virtual machine is given a virtual network interface that the card provides directly. For every such interface on the server, the card has a matching port (representor) on its own side, and the software switch running on the Arm cores connects those ports to the tenant networks.

The first packet of a new connection goes through the software switch on the Arm cores, which works out where it should go and which rules apply. That decision is then written into the card's hardware switch as a rule. Every later packet in the connection is handled by the hardware alone. The server's CPU never touches the traffic, and after the first packet neither do the Arm cores.

Diagram – Layout after NVIDIA's DPU mode architecture diagram in the BlueField Modes of Operation documentation.

Layout after NVIDIA's DPU mode architecture diagram in the BlueField Modes of Operation documentation.

What this gives you

The main gain is CPU. NVIDIA and Red Hat reported that moving the virtual network to the DPU cut server CPU use by 70% and freed eight cores per server. These are vendor figures, so treat them as an indication rather than a benchmark.

Virtual machines suit this design. They usually run for a long time, so the rules written into hardware stay useful. Each virtual network interface maps to one interface on the card, which keeps the setup simple.

The design also helps security. Network rules live on a device the tenant cannot reach, and the operator can run functions there, such as traffic filtering or monitoring, outside anything the tenant controls.

This matters most when a tenant is given a whole physical server. The tenant then has full administrator access, with no hypervisor in between, so anything enforced in software on that server can, in principle, be read or changed. The DPU is the one part of the machine the tenant cannot reach, which makes it the natural place to enforce the network.

What it does not solve

A DPU changes where networking runs, and nothing else about how tenants are managed.

Fast handover of a server from one tenant to another, snapshots and quick reboots come from the virtualisation layer, not from the network card. A physical server with a DPU still takes as long to wipe and reinstall as one without.

DPUs also introduce additional overhead. Each DPU runs its own operating system, which requires provisioning, regular updates, and ongoing monitoring. Consequently, a deployment of 100 DPU-equipped servers effectively introduces 100 additional computing environments that must be managed and maintained.

The tools

Software for running DPUs comes in layers, each automating the one below it.

At the bottom is configuring the software switch on each card by hand. This is useful for learning and testing, but does not scale.

Above that is OVN, an open-source project that manages software networks for many servers from one central control plane, used by OpenStack and several Kubernetes network plugins. OVN can run on the card itself, and some Kubernetes network plugins split their components between the server and the card.

At the top is NVIDIA's DOCA Platform Framework (DPF). DPF installs the operating system on each DPU, configures it, and deploys services onto it using Kubernetes. It runs the control plane of the DPU cluster as hosted control planes using Kamaji, the open-source project kMetal is built on.

With these tools, tenant networks can run on DPUs in two ways: a separate network control plane on the DPUs, or the platform's existing OVN control plane extended onto them. OpenStack has supported the second design, which it calls off-path SmartNIC DPUs, since its Yoga release in 2022.

Where kMetal stands

kMetal runs tenant Kubernetes clusters on bare-metal servers. Tenant workers can be virtual machines or whole physical servers. Each tenant gets its own private network, built on OVN, and its own hosted control plane. For virtual machine workers without a DPU, that network runs in software on each server, as described in the overlay section above.

kMetal uses BlueField DPUs to move this work off the server and enforce tenant network isolation there. This is part of the CLASTIX research and development effort. The effort is ongoing, through collaboration and research programmes with the University of Naples Federico II, and will bring further improvements.

Tenants never interact with the network implementation directly. They request networks, subnets and public addresses through kMetal's own API, which then creates the underlying OVN configuration. Whether the traffic runs through a server's CPU or through a DPU, tenants see the same networks and ask for them in the same way.

For physical tenant servers, the DPU is where isolation is enforced, and virtual machines and physical servers share a single network model.

Questions to ask about your own platform

If you run or are evaluating a shared platform, start with these questions:

  • How much server CPU does the network stack use under realistic load, and has anyone measured it?
  • Where are network rules enforced, and who can change them?
  • If a tenant is given a whole physical server, what stops them from changing its network configuration?
  • When a server moves from one tenant to another, how is its network state cleared and reapplied?
  • Is network configuration checked and corrected continuously, or applied once?
  • Can servers with and without DPUs serve the same tenant?

Sources

Get the good stuff first.

Get new releases, fresh thinking on Kubernetes, and upcoming events straight to your inbox.

We use Brevo for email communications. By submitting this form you agree that the personal data you provided will be transferred to Brevo for processing in accordance with both Brevo's Privacy Policy and our own CLASTIX Privacy Policy.

Opt-in *

You may unsubscribe at any time using the link in our newsletter.