This guest post is from Cumulus Networks. We thank Cumulus for being a sponsor.

400G Ethernet silicon is available and shipping from manufacturers such as Broadcom and Mellanox, and switches bristling with 400G ports are on the market. If it feels like 400G has shown up while you’re still grappling with 25 or 100G, it’s true. The current pace of Ethernet development is stunning.

According to a presentation from Cisco Live 2019, six new Ethernet speed standards were ratified between 2016 and 2018–that’s compared to six standards achieved over the previous 35 years.

This slide from the Ethernet Alliance shows the rapid progress that has occurred over the past few years (with 800G on the horizon!).

Such a torrid pace has been enabled by advances in manufacturing and mechanical engineering to produce the necessary optics, transceivers, and other components. It also doesn’t hurt that there’s a market for 400G among cloud giants, service providers, and even enterprises.

Whether for building high-density, high-bandwidth data centers or speeding transport between data centers, many organizations are keen to get 400G in place. The Dell’Oro group forecasts that 400G shipments will exceed 15 million ports by 2023.

400G And Open Networking

Organizations have to balance the benefits of greater throughput against the disruption, downtime, and potential service or operational impacts that come with swapping out old equipment for new. Traditional network vendors tightly couple switch software and hardware, which means if you want to embrace the next generation of Ethernet, you also have to embrace whatever software the vendor puts on the box–whether you want it or even need it.

For instance, a new 400G switch from a traditional vendor may come with a completely new operating system, or with software or features that may not be compatible with your existing tools and processes. This could hinder adoption and deployment of the latest hardware.

In addition, tight coupling may slow the pace of software innovation or new feature releases, because that software can only advance at the same rate as the network silicon, which has a significantly longer development cycle–years as opposed to months, or even weeks.

Open networking avoids these problems. You can upgrade from 10 to 40 to 100 to 400G devices without having to change your NOS or abandon the tools and processes you already use to run your network. In addition, open networking and the rise of merchant silicon means you get abundant hardware options from a variety of ODMs and OEMs, making it easier to find the right features and the right price.

Open networking also offers more choice when it comes to optics. Branded optics from traditional vendors are marked up exorbitantly, and if you swap in lower-cost third-party optics, your traditional vendor won’t support them.

By contrast, open networking gives you more choice. You can purchase any third-party optics that support the 400G standard and meet your cost and design requirements. Even better, Cumulus Networks will support them.

Cumulus Networks helps you leverage the benefits of open networking. Its Cumulus Linux network OS runs on dozens of platforms, and will support two 400G chipsets in early 2020. At the same time, your network operations team gets all the advantages of a robust, open ecosystem where new software and tools are being developed that easily hook in to your NOS because it’s Linux under the covers.

Key Details On 400G

For enterprise data centers, the biggest change with 400G–aside from the throughput–is optics. As of the first quarter of 2020, optics are scarce and expensive—approximately 2 or 3 times the cost of 100G.

Over time, as manufacturers ramp up production, those costs should come down, but you should anticipate that optics will account for a significant portion of the total cost of your 400G infrastructure.

QSFP-DD and OSFP

Two pluggable optics module types are available for data center access and interconnect use cases: QSFP-DD and OSFP.

Of the two, QSFP-DD has been built with backwards-compatibility in mind. If you buy a 400G switch that supports QSFP-DD, the ports on that switch can support QSFP+, QSFP28, and QSFP56.

In other words, if you have a 400G port, you can still plug a 100G or 40G optical module into that port and pass traffic through it. This backwards-compatibility gives customers more flexibility in their data center designs.

QSFP-DD has broad industry support, and is being actively promoted by vendors.

OSFP modules are also available, and the form factor is being used by some cloud giants. Because 400G requires more power than 100G, OSFP can draw up to 15W. Because of the greater power draw, the OSFP form factor is slightly wider and deeper that QSFP, and includes a built-in heat sink. This design means a special adapter is required for backwards compatibility with 100G QSFP.

Power and Cooling

400G infrastructure will draw more power than 100G. However, in most data centers, the real heat and power issues come from the servers, not the ToR switches or your leaf/spine boxes. As you plan a migration to 400G in the data center, be aware of the increased power draw, but it shouldn’t have a material effect on your power and cooling infrastructure or require significant changes or upgrades.

Cabling and Fiber

Last but not least, there are a number of cabling options for 400G based on the media type and supported distances.

Is 400G In Your Future?

Will 400G switches show up on your loading dock any time soon? That depends. While the first consumers of 400G are the cloud giants and large service providers, keep an eye on your own data center as a candidate.

For example, if you’re building out leaf-spine pods of 100G switches, 400G makes sense as an inter-pod spine. With a 400G inter-pod connection you can reduce your cabling, and have plenty of headroom to grow into.

400G can also be used to connect data centers. Rather than deal with the complexity of DWDM, you can use Ethernet and run fiber between the two locations.

Finally, if you’re already considering a data center refresh, keep in mind that applications and services have a way of consuming available bandwidth. Opting for 400G will give you plenty of room for growth.

Get Informed

To find out more about the benefits of open networking, check out this paper by Cumulus Networks, The Compelling Case for Open Networking.