You reduce it by using custom routing ASICs that can handle orders of magnitude more traffic than a CPU with equal or lower power consumption. To put things into perspective: Cisco’s latest routing ASIC — the Cisco Silicon One — can handle 10.8 Tbps (in fixed mode).
I’m sure Google and others have evaluated this, but it’s just kind of surprising that they opt to do per-packet processing in software.
10tb sounds like a big scary number but when you think about it that's only 100 100gb ports. It should be no surprise to you that a company making their own switches and NICs, where every RPC is individually authenticated and possibly also encrypted with session keys specific to the peer has no need of vendor "acceleration".
10 Tbps is not a scary number? I am honestly still in awe of how such a small box can route an entire city’s worth of traffic. Different perspectives I suppose.
Regarding your points: I am not sure I completely follow.
Firstly, as far as I know, Google does not make its own switching or routing ASICs.
Secondly, virtually all switching and routing ASICs are highly programmable. So if you need a custom protocol, you can implement it using the vendor’s microcode (e.g., P4). In other words, you are not limited to the protocols that the vendor has implemented.
Given the above, I don’t see what kind of technical requirements Google has that would disqualify the use of routing ASICs.
We obviously have very different perspectives on this issue. I think given Google's architecture, where a host handles millions of flows and they are all individually keyed, just the key management nightmare alone would be enough to discredit the idea of offloading crypto to an ASIC in a separate box.
but getting from 0..custom-asic is waay more harder than you make it out :o) plus you have a _custom_ stack on top of it, which is not malleable at all. moreover, getting a vendor lockin is probably not such a good idea anyways.
giving credit where it is due, intel's role in development of latest pcie-4.0 and upcoming pcie-5.0 specs and ofcourse the latest cpu's make it less and less appealing to consider putting up with custom h/w + s/w imho
with dpdk for example, doing 10g/core with 64b packet sizes is considered pretty standard these days. and if you don't want to roll your own, you always have companies like 6wind to help you (and ofcourse they share the code as well)
facebook/google etc. have, by now, probably at least a decade of experience running their own sdn enabled networks. facebook even has fboss to commoditize switch from white-label manufacturers...
SONiC is an interesting initiative by MS that reduces the impact of vendor lock-in by abstracting out ASIC specifics: https://azure.github.io/SONiC/
Even if you utilize every last cycle of CPU, the price/bps and energy/bit will still be way higher than almost any switching or routing ASIC on the market.
Does that actually accomplish the desired goal? Encrypting across data centers is one thing, but encrypting within a data center only makes sense if the threat model includes the datacenter's internal network being compromised – I suppose either physically or by hacking a router or something. I can't judge how much that threat model makes sense. But to the extent it does, a centralized router doesn't protect against it; in fact, if you're going to trust the router(s), there's no point encrypting the data at all. On the other hand, a per-server smart NIC would achieve equivalent protection, but maybe that's more expensive?
If you split secrets across NICs such that compromise of one NIC would only compromise a portion of internal datacenter traffic, then that would make sense. Again, as you noted, it depends on the threat model.
But from just the bandwidth and latency perspective, a custom ASIC makes more sense to me.
