Introduction, Setup, Test Method, Results Node Move, Closing Start

Prev

1

2

Next

End

Introduction

Update 10/3/16 - Added information about Ubiquiti 3x3

In my previous reviews of eero and Luma, I haven't done a very good job of testing their key feature: mesh. Mesh is this year's answer to all that ails your Wi-Fi networks' performance. Just buy a three pack of cute little devices for somewhere between $300 and $500 and you'll have the fastest, smoothest running Wi-Fi network ever...honest!

Mesh wireless systems rely on wireless connections among multiple nodes to work their magic. So our normal throughput vs. attenuation chamber testing doesn't tell the full story. So I took four products, eero, Luma and Ubiquiti Networks' Amplifi and Amplifi HD (yes, a full review is coming), set them up to force a multi-hop configuration, then measured performance at the root, middle and far-end nodes.

Setup

Mesh wireless makers would love buyers to set their systems up in a perfect configuration. This would be with the root node (the one hardwired to your modem or current router) in the center of your home and the two other nodes equidistant from the root node. And, oh yeah, make sure there is clear line of sight between everything.

This recommendation is pretty silly, given the main reason people need to resort to wireless extenders, BHR (Big Honkin' Routers) and now wireless mesh systems in the first place is to make up for the fact that they can't place their router in the center of their home. Of all the vendors trying to sell mesh wireless only NETGEAR acknowledges that most routers live at one end of a home, many times in the basement or lower level—the most convenient place for an installer to run a line. Their new Orbi assumes that the router unit will be stuck against a wall as shown in the promotion image below.

NETGEAR knows how the real world works

Since SmallNetBuilder is all about what happens in the real world, we tested all four products placing the root node in a downstairs office at one end of SNB Manor, the second node in an upstairs allway pretty much at the center of the house and the third node in our beloved kitchen zone of Wi-Fi death. The approximate locations are shown in the floorplan below.

Mesh test locations

The photos below provide a better idea of the node location surroundings.

Test location surroundings

Note the Hallway and Kitchen node locations appear closer together in the floor plan than they actually are. I used the same locations for all products, chosen to accommodate both wall-plugged and desktop formats and to ensure that I could test a true two-hop scenario. In other words, I didn't want the Kitchen node to be able to see the Office node.

I'll admit that better locations could have been chosen for each product. But none of the apps told me to move the nodes to improve the signal either during or after setup. In fact, only Amplifi provides any indication of signal strength between nodes or how nodes are connected.

The left screenshot shows Amplifi's two "mesh points" in single-hop mode, i.e. each directly connected to the router and with backhaul links of pretty much equal signal strength. The right screenshot shows the mesh points in a two-hop configuration, with a weak backhaul link between the router and first mesh point and a much stronger link between the first and second mesh point.

Amplifi shows you how things are connected

I think the right side Amplifi screenshot accurately illustrates the conditions all four products encountered; low signal level from Office to Hallway node and stronger signal level from Hallway to Kitchen node.

Before testing, I had eero and Amplifi products check for firmware updates. eero was running v 1.1.4 and Amplifi ran version 1.3.0. Luma provides neither a way to see firmware version nor a way to check for updates.

Test Method

I tried to use each system as a typical user would, i.e. not mucking with anything in the app other than what was needed to get set up. Basically this meant setting the network name (SSID) and wireless encryption key. In each case, I left other app settings at their defaults and set each system up with a single SSID for both bands. Only Amplifi lets you choose the band to use for backhaul (it defaults to 5 GHz) to both mesh points and whether to steer devices to the 5 GHz band (the default). I left both settings at their defaults. I made no attempt to figure out or control which band was used for backhaul during testing.

The test client was my trusty Lenovo x220i with a NETGEAR A6200 AC1200 class USB adapter plugged in and the notebook's internal wireless disabled. I checked the adapter's properties to ensure that both bands were disabled and no band preference was set. Any roaming settings were left at their default "auto" settings. In other words, I tried to have the client be as compliant as possible to any "guidance" the mesh system might provide for roaming or band steering. For each test, the laptop was location 5 - 6 ft. from the node being tested.

I used IxChariot running the throughput.scr test script with test file size set to 5,000,000 bytes and TCP/IP protocol; my usual setting for wireless AC testing. Each test was run once with a single connection for one minute and the average throughput recorded. Windows' wireless network connection status doesn't provide the BSSID of the wireless connection, i.e. the MAC address of the AP connected to. So I used NETGEAR's genie utility and recorded each connection's BSSID and link rate to confirm that the test client was connected to the proper node.

The Results

In general, the laptop connected to the 5 GHz band for all tests. The only exception was the Amplifi HD. When it roamed to the Hallway node, it initially connected in 5 GHz for the uplink test, which was run first. But before I started the downlink test, I noticed it switched to 2.4 GHz. So I reran the uplink test, then ran the downlink. The client stayed connected in 2.4 GHz for the Kitchen tests, too. But when I forced the client to reconnect, it connected to 5 GHz.

The first graph tells the tale for downlink throughput for each of the three test locations. The Kitchen - Reconnect results reflect a second test run in the Kitchen location, after forcing the client to disconnect, then reconnecting. This was necessary because in all cases, the test client always roamed from the Office to Hallway node, but did not roam from Hallway to Kitchen. To state more clearly, the Kitchen - Reconnect test is the one to look at for two-hop performance. The Hallway test represents best case single-hop performance.

The standard Amplifi's Office throughput is significantly lower than the Amplifi HD and also eero and Luma. But that is its only weak point. Otherwise, the throughput of the two is evenly matched, with the HD even falling behind the standard Amplifi in the first Kitchen run. The clear winner for downlink is eero. Although it doesn't turn in the highest throughput in the best-case Office run, it does in all the other tests, including the two-hop Kitchen - Reconnect test. For two-hops, the clear loser is Luma, barely eking out a connection at 0.7 Mbps.

Mesh throughput summary w/ Hallway node - downlink

Uplink shows a more significant difference between Amplifi and Amplifi HD once you get a hop or two in the connection. Luma again does not do well in the two-hop Kitchen - Reconnect test. eero again does well, but bows to the Amplifi HD in the two-hop test.

Mesh throughput summary w/ Hallway node - uplink