Like many OpenStack operators, we use Open vSwitch on our hypervisors. Unfortunately, when running Open vSwitch, you lose some ability to do some functions on the lower level physical interfaces on the box (packet flows do not pass through the regular netfilter path in the kernel, for example.) This has a side effect of LLDP tools (OpenLLDP in our case, which is packaged with CentOS) not being able to pick up LLDP frames received on the physical interfaces. Effectively this makes it impossible to use LLDP with Open vSwitch.

This has been a thorn in my side for some time, because we wanted to use the LLDP neighbor info to do some config management intelligence on the hypervisors. Basically, the box comes up, figures out where it is in the network (from LLDP) and then it knows how to configure itself with the right VLANs, bridge mappings, etc. I was only ever able to find some obscure mailing list threads relating to this. However, I recently stumbled across this thread, which gave me some new ideas.

Instead of listening on the physical interfaces for LLDP frames, an OVS port is added to the br-ext bridge for each physical interface. Then OVS flows are added to deliver the LLDP frames received on the physical interfaces to the corresponding OVS ports. OpenLLDP is configured to receive frames on the OVS ports and voila, we can now receive LLDP info.

ovs-vsctl add-port br-ext lldp.em1 vlan_mode=native-untagged -- set Interface lldp.em1 type=internal ovs-vsctl add-port br-ext lldp.em2 vlan_mode=native-untagged -- set Interface lldp.em2 type=internal

The resulting br-ext configuration:

Bridge br-ext Port "lldp.em1" Interface "lldp.em1" type: internal Port "mgmt0" Interface "mgmt0" type: internal Port "lldp.em2" Interface "lldp.em2" type: internal Port br-ext Interface br-ext type: internal Port "bond0" Interface "em1" Interface "em2"

Now to configure the flows in OVS, we will need to know the port IDs:

[root@hostname ~]# ovs-ofctl show br-ext 10(mgmt0): addr:5a:e8:29:cd:a2:c1 config: 0 state: 0 speed: 0 Mbps now, 0 Mbps max 40(em1): addr:00:21:9b:a5:3f:f1 config: 0 state: 0 current: 1GB-FD COPPER AUTO_NEG advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD AUTO_NEG supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG speed: 1000 Mbps now, 1000 Mbps max 41(em2): addr:00:21:9b:a5:3f:f3 config: 0 state: 0 current: 1GB-FD COPPER AUTO_NEG advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD AUTO_NEG supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG speed: 1000 Mbps now, 1000 Mbps max 42(lldp.em1): addr:fe:c8:37:40:30:e8 config: 0 state: 0 speed: 0 Mbps now, 0 Mbps max 43(lldp.em2): addr:6e:29:1f:d9:71:ca config: 0 state: 0 speed: 0 Mbps now, 0 Mbps max LOCAL(br-ext): addr:00:21:9b:a5:3f:f1 config: 0 state: 0 speed: 0 Mbps now, 0 Mbps max

Now, add the flows to OVS to deliver LLDP frames to the internal ports:

ovs-ofctl add-flow br-ext in_port=40,dl_dst=01:80:c2:00:00:0e,dl_type=0x88cc,actions=output:42 ovs-ofctl add-flow br-ext in_port=41,dl_dst=01:80:c2:00:00:0e,dl_type=0x88cc,actions=output:43

Flows are not persistent across reboots or Open vSwitch restarts, so to aid in that, you can add interface-up-scripts that run on boot when the interfaces are brought up:

/etc/sysconfig/network-scripts/interface-up-script-lldp.em1: #!/bin/bash PHY_PORT=`/bin/ovs-ofctl dump-ports br-ext em1 | /bin/grep -E '^[[:space:]]+port' | /bin/sed -r 's/^[[:space:]]+port[[:space:]]+([[:digit:]]+):.*$/\1/'` OVS_PORT=`/bin/ovs-ofctl dump-ports br-ext lldp.em1 | /bin/grep -E '^[[:space:]]+port' | /bin/sed -r 's/^[[:space:]]+port[[:space:]]+([[:digit:]]+):.*$/\1/'` /bin/ovs-ofctl add-flow br-ext in_port=${PHY_PORT},dl_dst=01:80:c2:00:00:0e,dl_type=0x88cc,actions=output:${OVS_PORT}

Enable receiving LLDP on the lldp.em1 and lldp.em2 ports:

lldptool set-lldp -g nb -i lldp.em2 adminStatus=rx lldptool set-lldp -g nb -i lldp.em2 adminStatus=rx

And shortly thereafter, we can query the LLDP TLVs on those ports:

lldptool -t -n -V PortID -i lldp.em1 Port ID TLV Local: Eth101/1/35 lldptool -t -n -V PortID -i lldp.em2 Port ID TLV Local: Eth102/1/35