Deploying new protocols without buying new switch silicon

In-field protocol deployment means adding support for a protocol, congestion algorithm, or telemetry format to network switches that are already racked and carrying traffic.

To carry out in-field protocol deployment, your switch silicon must meet a few requirements:

  • The silicon's packet-processing behavior must be defined in software rather than fixed at tape-out.
  • The silicon must have enough instruction budget per packet to absorb the new work.
  • The silicon must have a way to validate the change before it ships.

If these requirements aren't met, attempting an in-field protocol deployment may lead to disruptions in business operations: a cluster build waits on a congestion algorithm you cannot deploy, or a customer commitment slips because the telemetry to prove an SLA does not exist in your fabric. The capacity you have already paid for sits underused because the workload it was bought for needs transport behavior the hardware never had. The asset is fine. The timing is wrong.

That mismatch is structural. Infrastructure is approved on multi-year cycles because that is how capital works. Protocols emerge on the timescale of standards bodies and software releases. Every time the two fall out of step, someone has to choose between deploying late, deploying somewhere more expensive, or spending capital earlier than planned.

Programmability changes what that choice looks like.

If the packet-processing behavior is defined in software, a new protocol becomes an engineering task on hardware you already own rather than a procurement event. Three things follow:

  1. Time-to-capability drops from a hardware cycle to a development cycle.
  2. Deployed capital keeps earning, because the silicon can absorb a change it was not originally specified for.
  3. The decision moves from the finance calendar to your own. You deploy when you are ready, not when the next generation ships.

It is not free, and the rest of this guide is careful about where the argument stops. But the key distinction is simple: on fixed-function silicon, protocol change is a capital problem. On a programmable data path, it is a scheduling one.

Why protocol churn breaks the refresh cycle

Infrastructure is funded on a cycle, but protocols do not arrive on one.

Over the past few years, UEC has arrived to redefine Ethernet transport for AI clusters. RoCEv2 deployments have needed successive rounds of congestion tuning. QUIC has pushed transport logic into places that assumed TCP. New congestion algorithms show up faster than hardware can be specified, taped out, and deployed. Each one arrives expecting silicon that did not exist when the budget was written.

On fixed-function silicon, there are three responses to this expectation, and all three carry a cost:

  • You wait for the next generation, which means running without the capability for a year or more.
  • You deploy the function somewhere else, in software, on hosts, or on a separate appliance, which costs latency and power and is another thing to operate.
  • You bring the refresh forward, which is the expensive answer and the one finance remembers.

What makes this worse is that the cost is not one-off. Each protocol generation triggers the same decision, and the decisions compound. A team that has brought forward two refreshes in five years is not on a five-year cycle anymore, whatever the plan says.

Here's a tip: go back over the last refresh period and list every occasion when you needed a capability the silicon could not provide: a congestion algorithm you could not implement, a telemetry format you could not produce, an encapsulation you had to terminate somewhere else. For each one, write down what you actually did and what it cost. Most teams find the list is longer than they expected and that at least one entry was expensive. That list is the input to the economics later in this guide, and it is more reliable than any forecast because it already happened.

Four things that make a data path genuinely reprogrammable

Almost every vendor claims to offer programmable features, and the claim covers everything from a configuration API to an open instruction set. In this context, a “programmable data plane” means that the low-level packet-processing steps applied to traffic, such as parsing, matching and forwarding, can be redefined in software while still running in dedicated network hardware.

Four things separate vendor claims about programmability, and it's worth testing them on any supplier, including us. The first two criteria establish whether you can make a change at all. The second two establish whether the change will survive contact with production traffic. A vendor can pass the first pair and fail the second. That combination is the one that costs the most, because you discover the limitation only after you have committed.

One common way to describe this class of programmable switch is protocol independent switch architecture, or PISA: a switch design where the packet-processing pipeline is programmable enough to support new protocols and behaviours without redesigning the silicon.

Here are the four criteria for reprogrammable data paths to check with vendors.

1. The data path has to reach the packet-processing logic itself, not a layer above it

The test is what happens when you need something the vendor has never implemented. If the answer is a feature request, you have configurability. Real programmability means you can express new parsing and new forwarding behavior yourself, without the vendor in the loop.

2. The interface must be documented well enough to write against

An instruction set that is only reachable through a vendor compiler is a dependency wearing a different hat. Ask for the specification: encodings, semantics, register model, and the license under which it is available. The XISA specification document is published under the Mozilla Public License version 2.

3. The performance model must be predictable

Programmability that costs throughput is a trap because the cost lands after deployment. What matters is whether you can tell in advance what a change will cost. On an architecture with a stated instruction budget per packet, you can reason about a new feature before writing it. On architectures where added logic forces recirculation, you risk discovering the true cost only when traffic hits it.

4. Resources must not be locked to fixed stages

This is the constraint teams tend to hit the earliest and anticipate least. In classic pipeline architectures a table, counter, or register generally lives in one stage and can be touched once per packet because each packet passes through each stage once. Adding a second lookup against the same table means duplicating it or restructuring the program around the limitation.

Where memory is global and objects can be accessed more than once per packet, new logic can be added to an existing program instead of forcing a redesign.

Deployment examples with the Xsight Labs X2 switch

Both examples come from programs running on X2 switches today. In each case, an existing, working data plane is extended. Neither is a proof of concept.

Example 1: Adding validation to a forwarding program

Before: a program that parses IPv4 and forwards using a longest-prefix match. It works, and it will happily forward malformed packets into the network.

After: the same program with header validation, covering IP version, header length, TTL and checksum, plus TTL decrement and checksum rewrite. Packets that fail any check are dropped at ingress rather than leaking downstream.

The validation logic is eight instructions plus a set of branch targets:

BRBTSTCLR   R11.3, 1, not_ipv4_packetCMPI        R4.0, 28, 4, 4BRINEQ      ipv4_version_invalidCMPI        R4.0, 24, 5, 4BRILT       ipv4_ihl_invalidSUBI.F      R0.1, R4.2, 24, 8, 1BRILE       ipv4_ttl_too_smallSYNC.N      2, bad_ipv4_checksum

The point is not the specific checks. It is that changing what the switch considers a valid packet, a policy decision that on fixed silicon would be a feature request, is a bounded edit to a program, deployable to hardware already in the rack.

Full tutorial: IPv4 header validation and rewrite.

Example 2: Adding measurement to a program already in production

This is the one that matters most for congestion work, because congestion control begins with measurement, and the measurement you need is rarely the measurement you have.

Before: SNMP MAC counters. They tell you a discrepancy exists between packets in and packets out. They do not tell you why, and they cannot, because they count at the wrong layer.

After: per-ingress-port counters broken down by cause: good IPv4, non-IPv4, bad version, bad header length, bad TTL, bad checksum, lookup hit, lookup miss, counting both packets and bytes. Every drop is now attributable.

Two details are worth drawing out, because they are what in-field deployment actually looks like.

The parser is not touched at all. The packet formats have not changed, so the existing parser microcode stands. The entire change lives in the match-action program and in two JSON definition files. New capability, existing program, no redesign.

And the counter table is accessed twice for the same packet, once on validation, once after the lookup resolves to a hit or a miss:

ig_port_count:    ADD.SH      R0.2, R1.3, 0, 11, R0.0, 0, 11    COUNTER     RN, R5, R0.2, 4, 8, 2    ...    ADDI.SX.SH  R0.2, R1.3, 0, 11, 6    # LPM hit    COUNTER     RN, R5, R0.2, 4, 8, 2

On a stage-bound pipeline that pattern usually requires two separate counter tables, because a stage-local object can be reached once per packet. Here it is one table indexed twice. That is the difference between adding a feature and rebuilding a program to accommodate one. And it is the practical form the fourth criterion takes.

Full tutorial: Adding counter tables.

The amortization argument

If programmability lets one generation of silicon absorb protocol changes that would otherwise have forced a refresh, the economics change. That is worth doing as arithmetic rather than assertion, because for some deployments it will not hold. To check this, you'll need these four numbers to hand, all of which you should already have or can estimate.

Quantity
Where it comes from
A
Cost of a refresh cycle
Silicon, systems, installation, migration labor, the operational risk of touching a live fabric.
B
Protocol events per cycle
How many times in a refresh period you have needed a capability the silicon did not have. Look back, do not forecast.
C
Cost per event on fixed silicon
Whichever of the three responses you actually take: waiting, deploying elsewhere, or pulling the refresh forward.
D
Cost per event on programmable silicon
Engineering time to write, test and deploy a data-plane change. Not zero.

The comparison over one cycle is B × (C − D) against whatever premium the programmable option carries. If protocol events are rare or you absorb them cheaply already, the math does not favor programmability, and you should not pretend otherwise.

Suppose a refresh cycle is four years, and over the last four you have hit three capability gaps: one you waited out, one you handled on hosts at a measurable latency and power cost, and one that contributed to pulling a refresh forward by a year. If a pulled-forward refresh costs a quarter of A, and the host workaround costs you an ongoing operational overhead, then C is dominated by that single pulled-forward event, and B × (C − D) is close to a quarter of a refresh cycle, against D, which is a few weeks of engineering per event.

The sensitivity is worth noticing. The number that moves the answer is not the price of the silicon. It is how often you have needed something the silicon could not do, and what you did about it. Teams who have never pulled a refresh forward will find this argument weak. Teams who have done it twice will find it obvious.

What programmability does not solve

Programmability is not a general-purpose escape from hardware limits. It does not add bandwidth or ports, for example. A protocol you can implement in software still runs through the SerDes you bought. If the gap is capacity, this argument does not apply, and no amount of instruction-set access changes that.

The instruction budget is finite. Every feature you add consumes instructions per packet, and at small packet sizes the budget is tight. Programmability means you choose what to spend it on; it does not mean the budget is unlimited. A sufficiently expensive feature will cost throughput, and you should model that before you write it.

Table and memory capacity are physical. Elastic allocation makes better use of what is there, but it cannot create more.

The instruction set is specialized, and the gaps show. There is no multiply instruction in XISA, for instance: multiplication is rare enough in data-plane work that the silicon does not carry it. You work around it by precomputing values in the control plane, which is what the counters tutorial does. That is a perfectly good answer, and it is also a real constraint you will meet.

Remember, someone has to write, review, and test data-plane code. Most teams will work through P4 rather than at assembly level, and the tooling around any newer ISA is younger than the tooling around long-established platforms. That is a genuine cost, and it belongs in D.

The impact on the refresh cycle

Hardware does not last forever. Bandwidth demand still forces generational change, and it should.

What it changes is which pressures force it. On fixed silicon, both capacity growth and protocol change push toward the same expensive answer. On a programmable data path, protocol change stops being a hardware event and becomes an engineering one, which is the difference between a capital decision and a sprint.

The X-Series is built around that separation. The XISA specification document is published under the Mozilla Public License version 2, so you can write against it directly, and the roadmap is published rather than arriving one generation at a time, so capacity growth becomes a planned path rather than a surprise. UEC-ready programmable congestion measurement means the current wave of transport change is a software matter on silicon you can buy now.

Test the four criteria above on whoever you are considering. Then look at the two examples and ask what the equivalent change would cost you on the silicon you run today. The programmability evaluation checklist sets out ten criteria in the same spirit, and avoiding vendor lock-in covers the three dependencies that decide what leaving would cost.

Ready to get started?

Arrange a conversation with our solutions team to explore your infrastructure requirements and discover how open silicon can enhance your setup.