On my laptop, I have a VMWare virtual machine for testing. It is an Ubuntu Linux distribution with 20 GB of disk size and 512 MB of memory. Recently, I needed more Linux machines for simulating multi-router network environments. Because installing VMWare virtual machines is too much for the purpose, I decided to try the container-based virtualization supported by the Linux kernel – LXC. Below are the summary steps to add two new virtual machines on my ubuntu host with two network interfaces each.

Here is what the physical topology will look like:

And here is the logical topology after completing the configuration:

Before you start, make sure that your kernel version supports LXC. If it doesn’t, you may either change to a version that does or compile a custom kernel. The official LXC website website might be useful.

Besides the bridged network interface on my ubuntu connected to Internet I added two private network interfaces. Each of the new virtual machines has one interface bridged to the Internet and the other one to a private interface.

Now, we follow the steps below, executing all commands as root:

1. Install libvirt to do networking on it. You should not use bridge utils because bridging does not work on ESX VM.

apt-get install libvirt-bin

2. Install the lxc and debootstrap packages:

apt-get install lxc debootstrap

3. Create the capabilities dir and mount it:

mkdir /cgroup echo "none /cgroup cgroup defaults 0 0" >> /etc/fstab mount /cgroup

4. Edit the physical host network configuration (ubuntu) to allow the connections of the guests. First, take down the interface you wish to bridge:

ifdown eth0 eth1 eth2 lo

Then, edit the /etc/network/interfaces file and configure the physical interfaces as bridges. eth0 is connected to the Internet and eth1 and eth2 are virtual private interfaces. There is also one Loopback interface for experiments:

auto lo iface lo inet static address 10.0.0.3 netmask 255.255.255.255 auto br0 iface br0 inet dhcp bridge_ports eth0 bridge_stp off bridge_maxwait 0 bridge_fd 0 auto br1 iface br1 inet static bridge_ports eth1 bridge_stp off bridge_maxwait 0 bridge_fd 0 address 10.0.1.1 netmask 255.255.255.0 network 10.0.1.0 broadcast 10.0.1.255 auto br2 iface br2 inet static bridge_ports eth1 bridge_stp off bridge_maxwait 0 bridge_fd 0 address 10.0.2.1 netmask 255.255.255.0 network 10.0.2.0 broadcast 10.0.2.255

5. Get the interfaces up:

ifup br0 br1 br2

For more information on libvirt, you may refer the following URL:

http://wiki.libvirt.org/page/Networking#Debian.2FUbuntu_Bridging

6. Remove the network-manager so it does not mess with our settings:

apt-get remove network-manager

7. Uncomment the option for forwarding in the /etc/sysctl.conf file:

# allow ip forwarding net.ipv4.ip_forward = 1

And then, reload it:

/sbin/sysctl -p /etc/sysctl.conf

8. Now begins the configuration of our virtual devices. Unzip the lxc-debian script from /usr/share/doc/lxc/examples/lxc-debian.gz. Make it executable and open it for editing. Append the following lxc parameters to the copy_configuration() function before EOF:

# first public interface lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.name = eth0 lxc.network.mtu = 1500 # second private interface lxc.network.type = veth lxc.network.flags = up lxc.network.link = br1 lxc.network.name = eth1 lxc.network.mtu = 1500 EOF

eth0 is bridged to the Internet and is configured via DHCP, while eth1 is private and will be statically configured later.

9. Create your lxc dir. In this case, the name of my container is router1:

mkdir /home/lxc/router1

10. Create the root file system on /home/lxc/router1/rootfs and download Debian packages. Execute the lxc-debian script we’ve just edited above:

lxc-debian -p /home/lxc/router1 create

11. Create the container:

lxc-create -n router1 -f /home/lxc/router1/config

12. Run the container

lxc-start -n router1

The VM is run on the foreground to verify installation. If everything is OK, I stop it from another terminal:

lxc-stop -n router1

And then start it in the background:

lxc-start -n router1 -d

By default, you log in with root account and no password.

13. Install ping, iptables and your favorite text editor:

apt-get install iputils-ping apt-get install vim apt-get install iptables

14. Configure the hostname of the guest system. Edit the file /etc/hostname and change the name of the system to router1. Then run:

/etc/init.d/hostname.sh start

Disconnect and reconnect to the console for changes to take effect.

15. To connect to the new container you can use either the lxc-console -n router1 command or ssh to the configured IPs. After logging in, edit the /etc/network/interfaces file and add the following:

auto lo iface lo inet static address 10.0.0.1 netmask 255.255.255.255 auto eth1 iface eth1 inet static address 10.0.1.2 netmask 255.255.255.0 network 10.0.1.0 broadcast 10.0.1.255

Get the above interfaces up:

ifup eth1 lo

16. Install the BIRD routing daemon. Add the deb http://bird.network.cz/debian/ lenny main line to your /etc/apt/sources.list file. Then run apt-get update to fetch the new information from the repository an install the daemon with apt-get install bird. Now you have the newest version available. Do not install different versions of BIRD on the virtual machines unless you are looking for trouble :).

17. Enable IP forwarding on the guest system.

18. Repeat the steps for any other guest VM installations starting from step No 8.

Do not forget to install BIRD on the host system as well, as you will probably want it to be part of the topology as well.