(UDATE 2016-10-29: There is a catch in Ubiquiti's implementation in ER-X and ER-X-SFP as I discussed in a new blog post.)

Ubiquiti builds their Edgerouter X platform on MediaTek MT7621AT SoC. Currently this platform has three products: EdgeRouter X (5 GbE ports), EdgeRouter X SFP (ER-X plus one SFP port) and EdgePoint R6 (outdoor version of ER-X-SFP).

The EdgeRouter X platform has all GbE ports including SFP connected to the switch fabric inside MT7621AT. The switch is wired to MediaTek's high-speed Ralink bus. Below is a block diagram that partially describes the platform.

All ports in the switch fabric can be individually configured as independent ports or members of the hardware switch. As an independent port, EdgeOS creates a virtual interface for CPU routing. As members of the hardware switch ( switch0 in EdgeOS), any pair of ports forward packets to each other at wire speed bypassing CPU. On the CPU bandwidth, MediaTek says 2Gbps (1Gbps each way) maximum throughput between CPU and Switch. However, users found 3Gbps achievable in practice.

Different Configurations

Take the basic ER-X model as an example. It comes with five GbE ports which can be configured in multitude of possible ways. Easiest to begin with wizards in EdgeOS's GUI. Wizard WAN-2LAN2 will setup eth0 as WAN, eth1-eth4 as LAN when One LAN is checked in the wizard. In this configuration, eth1-eth4 form the hardware switch. When One LAN is not checked, eth1 is setup as LAN1 and eth2-eth4 as LAN2. eth2-eth4 becomes the hardware switch. Packets between WAN and LAN, LAN1 or LAN2 or between LAN1 and LAN2 have to route through CPU, potentially slowed down by the number of NAT, firewall and/or QoS rules as well as the max 3Gbps limit.

Using CLI, any port can be configured as WAN. Any combination of ports can form the hardware switch. E.g. eth0 as WAN, eth1-eth4 as independent ports which form a total of four LANs. Since throughput between CPU and Switch is limited to 3Gbps, this scenario does not make sense for top speed but it's possible.

Another example. eth0 as WAN1, eth1 as WAN2, and eth2-eth4 as a 3-port hardware switch. This is a dual WAN scenario which appeals favourably to SOHO users.

A more exotic example. eth0 as WAN1, eth1 as LAN1, eth2 as WAN2, eth3-eth4 as LAN2 and a 2-port hardware switch. Connect WAN1 to LAN1, and WAN2 to LAN2. We have one router serving two separate LANs each with its own WAN connectivity. We can apply some sort of fail-over/load balance among the two pairs of WAN-LAN. Additionally can hook an external switch to LAN1 for more clients.

No scenario is bad if it serves a purpose in a specific usage. Just observe the max throughput between CPU and Switch for optimal routing speed.

Bug in the Boot Loader

There was a bug in earlier ER-X models where on boot and before completion the switch fabric's initial state is a hardware switch of all ports. If people have complex network setup behind a ER-X, imagine nasty things happen as packets will pass through the switch freely. The bug is already fixed with a boot loader update available here. The fix initialises the switch fabric as disconnected ports and re-configures the ports according to system preference when Linux kernel loads.