Introduction & Specifications, Pricing, and Availability

VIEW GALLERY - 24 IMAGES

The high-speed interconnect market is going through very exciting changes. While recent headlines promise 100Gbps connections (some companies are already shipping), it's important not to count out established technologies that are also going through changes.

We've talked a lot about 10GbE technology with product reviews on switches and network interface cards over the past two years. 10GbE deployment costs are shrinking. ASRock just announced the first workstation class motherboard with onboard 10GbE, a product that costs as much as a single 10GbE NIC did just a few months ago. Over the next year, we'll continue to see 10GbE prices slip, but that isn't the only exciting trend.

Intel recently released the company's successor to the X520 / X540 10GbE controller products. The new XL710 (40GbE) and X710 (10GbE) controllers use less power than the products they replace, and the XL710 40GbE controller uses less power than any other 40GbE card on the market today. This is big news for datacenters where power consumption from thousands of servers dominates Opex.

Supermicro is no stranger to datacenters, so it comes as no surprise that the company is among the first to release a product based on Intel's new XL710 network controller.

Specifications, Pricing and Availability

At the time of writing, Supermicro doesn't have an official product page for the AOC-S40G-iQ2 online, and channel availability is limited. This will change over time as the company ramps up production, and channel partners start to carry the new dual 40GbE NIC.

The specifications sheet covers the basics, but as you'll see, there is more to the Fortville controller than the initial specifications show.

Based on Intel's XL710 'Fortville' controller, the AOC-S40G-iQ2 is a PCIe 3.0 x8 (eight-lane) 40GbE network interface card with two QSFP+ ports. The card is half height and half length (HHHL) for use in tight environments. An assortment of standard features are listed on the specifications, such as iSCSI remote boot support, and Fibre Channel over Ethernet. Things get interesting when you get into the full feature list.

With an eye towards next generation datacenters, the Supermicro AOC-S40G-iQ2 supports technologies geared toward virtualization, like stateless offload VXLAN, NVGRE, and Geneve for overlay networks. These technologies now make it possible to distribute network traffic to many CPU cores.

For today's review, we'll use the Intel XL710 controller spec sheet for the full list of features.

The new features are nice, and will increase the density possible for virtual environments, but the greatest feature is the power consumption. The new Supermicro 40GbE NIC should use around 3.6 watts under typical workloads, and see peak power at just 4 watts. This is a large reduction from Intel's previous generation 10GbE network controllers that used around 14 watts for dual 10GbE.

Supermicro AOC-S40G-iQ2

Here we get our first look at the Supermicro AOC-S40G-iQ2 40GbE NIC. The card uses passive cooling for the low-power Intel controller.

Most of the surface mount components are on the top side of the drive, but a few are on the bottom side. A reference guide (top left) shows the different settings for the DIP switches, but we will discuss that in greater detail later on.

The -iQ2 model has two QSFP+ ports. I wouldn't be surprised if Supermicro released a single QSFP+ model, and even a model based on the X710 chipset for 10GbE, but both are unconfirmed at this time. The ports on the unit we're testing today support both QSFP+ optical MPO, or CR4 (direct attached copper).

Four openings on the half height backplate allow air to move past the card and exit the system case.

The NIC controller is cooled by a passive heat sink. Currently, we don't have any recommended fair flow numbers, but the card stays relatively cool compared to other 40GbE solutions and Infiniband cards we've tested, even under load.

The card interfaces with the server through a PCIe 3.0 eight-lane connector. It's also backward compatible with PCIe 2.0 for legacy installations.

This is the Intel XL710 chip. The package covers the inner wiring, but the main silicon is exposed, and comes in direct contact with the heat-sink-like second generation Pentium III processors. The die size is about the same size as Coppermine Pentium III processors as well.

Supermicro used DIP switches on the AOC-S40G-iQ2 to give users control of SMBUS, and to enable or disable the thermal reading. This is a feature we didn't find on Intel's Converged Ethernet adapter based on the same XL710 chip.

This chart was sent to us by Supermicro, and it details the DIP switch settings. For our testing, we used SMBUS ARP mode, and left the thermal reading enabled; these are the default settings.

Inside the Configuration Menu

We used Intel's XL710 chipset drivers to test the Supermicro AOC-S40G-iQ2. The latest driver for Linux is 1.1.23, and the latest driver for Windows-based systems is ProWare 19.5.

The driver comes with preconfigured profiles of the optimize settings based on different server scenarios. You can also build your own custom profile by digging down into the individual settings to tune performance. The preloaded profiles are detailed below.

Standard Server - This profile is optimized for typical servers.

Web Server - This profile is optimized for IIS and HTTP based web servers.

Virtualization Server - This profile is optimized for Microsoft's Hyper-V virtualization environment; selecting this profile disables Data Center Bridging (DCB).

Storage Server - This profile is optimized for Fibre Channel over Ethernet, and iSCSI over DCB performance. Selecting this profile will disable SR-IOV and VMQ.

Storage + Virtualization - This profile is optimized for a combination of storage and virtualization requirements.

Low Latency - This profile is optimized to minimize network latency.

The Performance Options tab opens another set of settings.

The drive allows users to configure teaming with two or more ports from Intel NIC chipsets.

Test System Setup and Testing

Test System Setup

For our test today, we're using two dual Xeon systems, each with a Supermicro AOC-S40G-iQ2 40GbE NIC installed. We're testing through a Supermicro SSE0X3348TR switch with the latest firmware, and the AOC-S40G-iQ2 cards connected to the switch's 40Gbps Ethernet ports. The test software is iXChariot from IXIA.

Supermicro SSE-X3348TR 10GbE / 40GbE Switch

With a switching fabric of 1284 Gbps through forty-eight 10GbE ports, and four 40GbE QSFP+ connectors, the SSE-X3348TR is our switch of choice for testing SMB and enterprise network products.

Benchmarks - Throughput

We used the industry standard IXIA ixChariot to test the Supermicro AOC-S40G-iQ2. In this first test, we used two Supermicro dual Xeon systems, each loaded with an AOC-S40G-iQ2, and connected to a Supermicro SSE-X3348TR switch.

With sixteen instances of iXChariot's Ultra High Performance profile test, we observed a one way transfer speed of roughly 36.8 Gbps on average.

Final Thoughts

The phrase most often used when speaking of Intel's new XL710 network controller is: game changer. We have to agree. The reduced power consumption and higher throughput performance increase server density.

Paired with Intel's new Haswell Xeon processors that double Sandy/Ivy Bridge performance, and Intel's DC P3700 NVMe storage, datacenters are now able to reduce the number of servers used to meet current workloads at a two to one ratio. This assumes datacenters move from E5 2690 to E5 2699 V3 systems. The Capex investment is recovered through Opex in the form of reduced power consumption and rack space. Another solution is to simply grow the datacenter.

Outside of the traditional datacenter, businesses with a large group of servers can also benefit from the new Supermicro XL710 controlled 40GbE NIC. Again, this is paired with the latest generation of hardware from Intel, such as processors with an increased core count and faster storage technologies. Businesses can reduce the number of servers with virtualized systems (one of the key benefits of the Supermicro AOC-S40G-iQ2), and reduce operating costs associated with cooling and system power.

At the time of writing, we weren't able to find the AOC-S40G-iQ2 in the channel in America, but that should change soon. Supermicro doesn't have an official product page for the part yet either, but we did find a reference to the NIC on one of the new X10 servers as an optional component. When available, we'll update this review with a product page and pricing.