We examine the fields within an Ethernet packet and discuss how they are processed by an open source FPGA network processor.

Overview

This article reviews the structure of an Ethernet packet and basic packet processing in the context of the Private Island FPGA-based open source network processor. This article also discusses a subset of the basic Ethernet MAC controller functionality provided by the project's mac controller module.

The project's source code is hosted here using cgit, and here is a link to mac.v.

The Private Island open source project provides a strong foundation for network security and privacy by enabling developers to directly and deterministically inspect, classify, and filter packets on a network without relying on layers of opaque abstraction provided by closed source firmware and proprietary finite state machines (FSM) executing on custom silicon.

This article relies primarily on the IEEE Standard 803.3-2018 for definitions and terminology. Note that this specification is available for free from IEEE via their GET Program, and the reader should refer to this standard for additional details.

Ethernet Frame vs. Packet?

The difference between an Ethernet packet and a frame can be confusing. IEEE 802.3, section 3, defines an Ethernet MAC frame as consisting of a Destination Address, Source Address, Length/Type field, MAC Client Data, Pad (if required), and Frame Check Sequence. You can see these fields below in Figure 1.

A Ethernet packet consists of an Ethernet MAC frame as defined previously, preceded by the Preamble and a Start Frame Delimiter (SFD). An Extension (EXT) field follows the Ethernet frame for 1000 Mb/s half duplex operation. This is what is transmitted on an Ethernet wire in a home, lab, office, or factory floor.

As as you probably know, there is also an IP (Internet Protocol) packet. This is defined at a higher layer than Ethernet and is transmitted in the data field of the Ethernet frame.

Figure 1. Ethernet Packet

Fields within an Ethernet Frame

The destination and source addresses enable proper switching of Ethernet frames on an Ethernet local area network (LAN). These addresses are typically referred to as a "MAC address". IEEE defines the first two bits of an address to specify additional information. In practice, the two bits only apply to the destination address and only the first bit is meaningful (signals that the packet is destined for a group or individual / unicast station).

A broadcast destination address consisting of all 1's (ff:ff:ff:ff:ff:ff) signals that all stations on a LAN should receive the Ethernet packet. Note that broadcast is a common occurrence on a typical LAN for operations such as announcements and higher layer address discovery (e.g., ARP). However, such activity can have negative security consequences, and this is something that can be easily filtered / dropped using this system.

The Length/Type field specifies three types of MAC frames:

basic frame

Q-tagged frame (VLAN)

envelope frame

If the value in the Length/Type field is less than or equal to 1500, then the field indicates the size of the data field for a basic frame. If the value is greater tha 1636 (0x600), then the field indicates an Ethertype (e.g., 0x08000 for IPv4 and 0x86DD for IPv6).

Maximum and minimum field lengths are specified in 802.3 Table 4-2: "MAC parameters". For example, a minimum and maximum basic frame size is 64 and 1518 bytes1 respectively. Subtract 18 to calculate the minimum and maximum data field size. If the data field is less than this minumum parameter, then a pad field is required.

Note that the aforementioned MAC Client Data is often an IP packet carrying user data, such as web requests or voice & video.

The Frame Check Sequence (FCS) field carries a cyclic redundancy check (CRC) generated by the transmitting Ethernet station and checked by the receiver to detect potential bit errors that occurred during transmission (e.g., noise on the wire).

1) This article denotes 8-bit data as bytes. The IEEE specification refers to these as octets.

SGMII Framing

Private Island currently utilizes a Serial Gigabit Media Independent Interface (SGMII) bus for communication between the FPGA's MAC layer and external Gigabit Ethernet PHYs. If you're unfamiliar with SGMII, please see our "Review of Ethernet SGMII Concepts" article.

Figure 1 shows SGMII framing & control fields encapsulating the Ethernet Packet. These fields are defined in IEEE 802.3, section 36. Note that IEEE refers to these as special code groups.

/S/: Start_of_Packet

/T/: End_of_Packet

/R/: Carrier_Extend

These fields are present inside the FPGA & on the SGMII bus (between FPGA and PHY) but not on the Ethernet network. They are used by the FPGA's state machines to frame the Ethernet packet, and the mac module distinguishes these fields from data using the rx_k signal (asserted for special code groups)

The figure below shows a subset of the FPGA modular architecture with the mac module in the center. Among other duties, this module implements the front-end transmit and receive state machines for processing the packet fields shown in Figure 1. These functions can also be referred to as a subset of MAC controller functionality.

Figure 2. mac module

The mac module interfaces on the PHY side with a hard SERDES/PCS block provided by the Lattice ECP5UM FPGA. This block performs parallel to serial conversion and framing & signalling required to communicate on the SGMII bus. Additional details about this block can be found in the article: "ECP5 PCS/SERDES Architecture for FPGA Open Source Network Processor".

The blocks shown to the right of the mac module are open source Verilog modules. We will refer to this side of the figure and these modules as the soft side to distinguish it from the hard SERDES/PCS block.

On the soft data receive side, the mac module asserts signals and passes data to the pkt_filter and drop_fifo modules. The pkt_filter module contains a content addressable memory (CAM) that is used to compare data fields (e.g., address) in the Ethernet frame against the CAM contents to determine how to handle the packet (e.g., drop it). The drop_fifo module provides temporary storage to hold the Ethernet frame contents until a decision is made on whether to pass the frame into the switch or drop it. Both modules depend on signals from mac derived from its state machines and logic.

On the soft transmit side, the mac module interfaces directly with the Ethernet switch module (switch), and the transmit state machine provides the required framing to transmit the Ethernet packet onto the network.

A View inside the FPGA

Shown below are two screenshots from the Lattice Diamond Reveal virtual logic analyzer to help further convey the information in this article and show a portion of the tool environment when developing with the Private Island system. The capture was made using the Darsena development board, and the signals shown are the actual Verilog nets and registers within the mac module. The view was triggered by the assertion of the rx_fifo_we signal, which is the write enable for the drop_fifo module.1

Both figures show the start of a received Ethernet packet (refer back to Figure 1). An IPv4 packet is being received in Figure 3, and an ARP packet is being received in Figure 4. The rx_state register holds the state of the Ethernet packet receive state machine. The signals shown below rx_state are all derived from the receive state machine and are passed to the other soft modules to help control the decisions regarding how to process the received Ethernet packet.

Figure 3. mac receives an IPv4 packet

Figure 4. mac data receives an ARP packet

Summary

New functionality is actively being developed for this project, and the mac module is likely to undergo substantial changes, such as being divided into a MAC controller and upper layer state machines.

Note that the Private Island Git repository has recently been removed from Github and will be relocated to mindchasers.com before the end of March 2019 using CGit.

Footnotes