11 March 2015 Ben Blair

Open Compute 6 Pack

Facebook’s Open Hardware Modular Switch and The Future of Scalable, Flexible Networking

by Ben

Over the past few years, Facebook has been reinventing their datacenters from the ground up, and sharing their resultant hardware designs through OpenCompute. Facebook started with a server, then added a top-of-rack switch called Wedge. Just last month, the Internet giant put the finishing touches on the final networking component, an open hardware modular switch called “6-pack".

With 6-pack, Facebook has created an architecture that enables the company to build a network fabric of nearly any size using a simple set of common building blocks. Those open hardware building blocks include a rack with integrated power backup and redundant distribution, server, storage, top-of-rack switch, and now core fabric switch. Many of these individual components are already available from vendors such as Hyve, Penguin Computing, Quanta and now HP.

Why an open network?

According to Facebook, the key driver behind an open network infrastructure is full operator control. This is something every IT infrastructure operator requires in order to scale quickly and gain efficiencies across the network.

What does full operator control look like in practice? To illustrate, let’s say you want to use Chef to orchestrate your network configuration. If you're running a proprietary hardware platform like Cisco or Juniper, you have to wait for the OEM to add support for Chef. You might be waiting for years. But if each of your network devices is a Linux server that you control, you're not stuck waiting for anyone! You're in the driver's seat, not the OEM.

If the pace that you scale your business is beholden to the release schedule of a specific OEM, you’ve not only put yourself at a competitive disadvantage, but also stifled innovation and growth. According to ISI Technology Research, 80% of a network engineer's time is spent manually configuring the network. It stands to reason that modern devops practices applied to networks running some flavor of Linux will dramatically reduce that percentage.

The Cost Savings are Compelling

If saving engineers more time isn’t enough to justify an honest look at open, modularized components, consider the cost savings. Cumulus Networks published a CapEx Analysis for bare metal switches with a Linux operating system on top of them. In this brief, the cost comparison used a 10-rack leaf-and-spine setup as the example.

List price for networking hardware supporting an Open Compute Project (OCP)-compatible setup from Quanta was $96,179 vs. $755,600 from Cisco and $641,028 from Arista. Even if you add $44K for Cumulus’ 3-year support package, the price differential is still enormous. There is ample room to extract infrastructure savings by replacing incumbent solutions with bare metal switch solutions. These cost savings can then be reinvested into more hardware and other innovations.

Going open is even more compelling when you take into account the OpEx savings.

The Future is Soon, If Not Now

Most businesses are not grappling with the scaling issues that prompted Facebook to develop its own open network. But if history is any guide, the approach will percolate down to all-sized datacenter operators - not just the giants. Facebook will propose that its new 6-pack switch design becomes part of the OCP, and they have committed to continue working with the OCP community to develop open network technologies that are more flexible, scalable, and efficient. However, it's far from certain that OCP will be the winning standards body.

There’s more than a hint of déjà vu underway if you consider what happened in the server market. The same unbundling that swept the server market resulted when OS was separated Hardware. That trend left the server market a low-margin commodity which is exactly where networking is headed. There are strong arguments that it will happen faster with networking because the current pain is greater. As AWS has said, the network is a bigger pain point than servers.

Modular Flexibility #FTW

Looking ahead, there is clearly an optimal state from the end-user point of view. It’s a world where you can order hardware from one of many super-scale manufacturers like Quanta based on open-sourced designs. The lure and ease of shopping around and vendor-hopping will be strong. You can have an integrator build out and deliver your racks to the datacenter space you rent from an established provider like Switch or Equinix. You will run open-source, Linux-based software on everything including server, networking and storage. And you can orchestrate everything with whatever open-source tools that please you at the time (today that might be something like Mesos or Kubernetes + Chef).

There’s no denying that 6-pack, the final component in Facebook’s open network vision, has added fuel to a shift in how we view hardware that will continue to reverberate throughout the tech world. If the end result is faster, more efficient infrastructure and undeniable savings to the bottom line, it’s hard to be anything but bullish on the trend.

For members trading on the MarkITx Exchange, the rapid adoption of open hardware means increasing commoditization and fungibility. And that means more consolidated liquidity which is good for every buyer and seller in the market.

Filed Under: Tech Community