KVM - The Linux Kernel-Based Virtual Machine
News, Blogs and Resources on the Linux (KVM) Kernel-Based Virtual Machine

Tip: How to Get Maximum Network Performance using paravirtual drivers and bridged networking

In this article I will outline how to configure bridged networking to a tap interface on a Fedora KVM host as there seems to be a lack of good documentation on how to do this. It also turns out that bridged networking using a tap interface provides much better network performance compared to userspace networking. Combine paravirtual drivers with bridging to a tap interface and you should expect near native network performance. So let's begin.

 

 

Install Fedora Packages

There are two packages that you will need to install on your Fedora host; bridge-utils and openvpn. You can install these using the following commands.

# yum -y install bridge-utils
# yum -y install openvpn

The openvpn package actually provides the tap interface which will be added to the bridge. After installing openvpn package, verify the tun device exists with the following command.

[root@localhost network-scripts]# ls -la /dev/net/tun
crw------- 1 root root 10, 200 2008-04-09 11:53 /dev/net/tun

 

 

Create Virtual Bridge

Next you need to create the virtual bridge to which the physical and the tap interface will be added. We create the virtual bridge with the following command

# /usr/sbin/brctl addbr br0

At this point we can verify that the bridge has no interfaces attached to it using the brctl command:

[root@localhost network-scripts]# brctl show 
bridge name     bridge id               STP enabled     interfaces 
                   br0             8000.000000000000       no 

The output shows the bridge called br0 now exists and a randomly generated bridge-id. Also note that there are no interfaces attached to the bridge. STP is not important and should not be enabled. STP refers to spanning tree protocol and is a protocol used for loop prevention in a switched network when there are redundant links between switches. Since you have only one bridge here it doesn't apply. At this point the bridge device will show up in the Network management GUI tool. To view the device in the GUI, type the command system-config-network.

[root@localhost network-scripts]# system-config-network

When you type this you should see the bridge device br0 as shown below.

 

 

Add the Physical and tap interfaces to the bridge

Next you need to add the tap and physical interface to the bridge. There are two interfaces that you typically want to add to the bridge. These are

  • eth0 ( or whatever physical interface your VM will be bridging through ) and
  • tap0

Before we can add the tap interface, you first need to create it. Create it using the following command.

[root@localhost network-scripts]# openvpn --mktun --dev tap0 
Wed Apr  9 20:22:49 2008 TUN/TAP device tap0 opened 
Wed Apr  9 20:22:49 2008 Persist state set to: ON 

Now set the tap interface to an up state using the following command.

[root@localhost network-scripts]# ifconfig tap0 up 

Verify that it is up with the following command

[root@localhost network-scripts]# ifconfig tap0 

tap0      Link encap:Ethernet  HWaddr 00:FF:B2:54:F1:31   
          inet6 addr: fe80::2ff:b2ff:fe54:f131/64 Scope:Link 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1 
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0 
          TX packets:0 errors:0 dropped:6 overruns:0 carrier:0 
          collisions:0 txqueuelen:100  
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b) 

Now add both the interfaces to the bridge as follows

[root@localhost network-scripts]# brctl addif br0 eth0 
[root@localhost network-scripts]# brctl addif br0 tap0 

Now when you issue the brctl show command, you will see the two interfaces added to the bridge

[root@localhost network-scripts]# brctl show

bridge name     bridge id               STP enabled     interfaces 
br0             8000.0019b97ec863       no              eth0 
                                                        tap0 

 

 

Configure Startup scripts

There are three scripts that need to be modified . These are

  • /etc/sysconfig/network-scripts/ifcfg-br0
  • /etc/sysconfig/network-scripts/ifcfg-eth0
  • /somelocation/qemu-ifup 

The last script, qemu-ifup can be placed anywhere you like on the filesystem. I placed this script in my home directory. Lets configure the br0 script first. Create the file /etc/sysconfig/network-scripts/ifcfg-br0 with the following contents:

DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes

The eth0 script should be automatically configured and yours should look like the following. The important line to note is the line with BRIDGE=br0.

# Broadcom Corporation BCM4401-B0 100Base-TX 
DEVICE=eth0 
BRIDGE=br0 
BOOTPROTO=dhcp 
HWADDR=00:19:b9:7e:c8:63 
ONBOOT=yes 
TYPE=Ethernet 
USERCTL=no 
IPV6INIT=no 
PEERDNS=yes 
NM_CONTROLLED=no 

Next create a script called qemu-ifup with the following contents. Make a note of where you store this file as it will be used as an option during qemu/kvm startup.

#!/bin/sh 
 switch=$(/sbin/ip route list | awk '/^default / { print $NF }') 
/sbin/ifconfig $1 0.0.0.0 up 
/usr/sbin/brctl addif ${switch} $1 

Now activate your br0 using the GUI, activate and save your configuration. It will prompt you to restart your computer or network configuration. Restart your network configuration.

[root@localhost scripts]# /etc/rc.d/init.d/network restart 
Shutting down interface br0:                               [  OK  ] 
Shutting down interface eth0:                              [  OK  ] 
Shutting down loopback interface:                          [  OK  ] 
Bringing up loopback interface:                            [  OK  ] 
Bringing up interface eth0:                                [  OK  ] 
Bringing up interface br0:   
Determining IP information for br0... done. 
                                                           [  OK  ] 

Your br0 interface should show an up state when network script is restarted as shown above. Now if you type an ip route list command, you should see output similar to below.

[root@localhost scripts]# ip route list 
172.16.26.0/24 dev br0  proto kernel  scope link  src 172.16.26.110  
169.254.0.0/16 dev br0  scope link  
default via 172.16.26.1 dev br0  

Note the default route via br0.

 

Generate MAC address and startup KVM Virtual Machine

Now before you can start kvm to use bridge networking, you need to generate a random mac address to bind to your virtual machine's nic. I use the following script to generate a random mac-address.

#!/usr/bin/python
# macgen.py script to generate a MAC address for Red Hat Virtualization guests
#
import random
#
def randomMAC():
	mac = [ 0x00, 0x16, 0x3e,
		random.randint(0x00, 0x7f),
		random.randint(0x00, 0xff),
		random.randint(0x00, 0xff) ]
	return ':'.join(map(lambda x: "%02x" % x, mac))
#
print randomMAC()

The script called macgen.py is borrwoed from Red Hat in generating random mac addresses and you can execute it as follows:

[root@localhost scripts]# ./macgen.py
00:16:3e:75:09:aa

As you can see the script generated the random mac address 00:16:3e:75:09:aa. You will use this mac-address in your qemu start script for bridged networking. Below is an example of my startup script.

/usr/local/kvm/bin/qemu-system-x86_64 \
-hda windowsxp.img \
-m 512 \
-net nic,macaddr=00:16:3e:75:09:aa \
-net tap,script=/home/user/software/scripts/qemu-ifup &

Two things to note in this start script are the mac-address and the reference to the qemu-ifup script that you configured earlier. The above script still uses an emulated nic. In order for maximum performance, configure your kvm startup script to use paravirtual drivers with bridged networking as follows:

/usr/local/kvm/bin/qemu-system-x86_64 \ 
-hda windowsxp.img \ 
-m 512 \ 
-net nic,model=virtio,macaddr=00:16:3e:75:09:aa \ 
-net tap,script=/home/hsolomon/software/scripts/qemu-ifup & 

Your virtual machine will now start in bridged mode using paravirtual device drivers if you already have them installed in your virtual machine. To see how to install paravirtual drivers see related posts here. Now you can enjoy near native virtual machine networking performance. macgen.py qemu-ifup ifcfg-br0

See Also

Comments

tunctl vs openvpn mktun

tap interfaces I created with openvpn --mktun didn't work at all.
perhaps it's me, but tuncl (from user mode linux tools) worked instead.
hope this will help people in same case than me.

You're quite correct. I

You're quite correct. I was unaware that Fedora also supported tunctl now and it is actually simpler to use than openvpn. Install tunctl by issuing the command 'yum -y install tunctl'. Then to create tap interface simply type 'tunctl' . Everytime you type 'tunctl', it creates a new tap interface and increments the number ie. tap0, tap1, tap2...etc

in fact, you don't need to

in fact, you don't need to bother with persistent pre-created tap interfaces at all
if you don't need.
qemu will create and destroy tap interfaces on the fly starting with the next available tap
name.
you can optionally select the name with ifname :

-net tap,ifname=tap0

Haven't tried it but would

Haven't tried it but would that would simplify setting up the tap interface. I seem to recall that Fedora never supported the tunctl or uml-utilities packages in its official repository so this must be a recent development. Thanks for the info.

Haydn ... stumbled upon your

Haydn ... stumbled upon your page looking for info on Bridging in Fedora 9.

... wondering if you have tried using tunctl or openvpn in conjunction with Network Manager on Fedora 9?

As far as I can tell, Network Manager refuses to allow a Bridge setup. I found a reference on the Fedora Virtualization ToDo List here: http://fedoraproject.org/wiki/VirtualizationTodoList that seems to indicate there is a problem using Bridge Networks + Network Manager.

But no matter what, it would be great to read your steps using tunctl instead of openvpn to achieve a paravirtual drivers with a bridge network.

many thanks

Hi David, It did cross my

Hi David,

It did cross my mind to do another post using tunctl instead of openvpn after laurent pointed out that tunctl is so much simpler. I will investigate the link you sent and do another post detailing this procedure with tunctl.

but what about virt-manager?

Nice post, but now how do I incorporate that into virt-manager/libvirt so that I have something more than command line control of my VMs using the bridge? Is there extra XML-foo to be added to the VM config file?

Alex,

Hi Alex,

Check out the link that David posted. Fedora plans to add management of the bridge to it's Network managemnt GUI tool. I really haven't looked into the GUI tools  yet. 

MAC generator script

I used your MAC generator script while experimenting with Qemu/KVM, and found there to be two problems with it.

1. You call /bin/sh, but $RANDOM doesn't necessarily work in /bin/sh, although it does when sh is actually bash. First line then should really be #!/bin/bash. I found this out the hard way on ubuntu, which is apparently one of the few distros not considering the two equal.

2. Any MAC which starts with an odd number (including DE) is a multicast MAC rather than a unicast MAC. Qemu/KVM (at least in the version I'm using) catches this by setting the first byte to 00 when you give it a DE:AD:BE:EF:xx:xx address, resulting in 00:AD:BE:EF:xx:xx, but it's not necessarily always obvious why that happens.

Re: mac generator script

Jasper,

I will update script and provide an update. Thanks for pointing out this detail. Good catch.

Updated MAC?

Where's the updated MAC generator?

Re: updated mac script

alphadogg,

I've updated the mac script using a borrowed python script from Red Hat and modified the blog post. Note that the first 3 octets are owned by organizations, the one used in the script is owned by Xensource. This should not be a problem if you're using xensource virtual machines on same LAN because the probability of a conflict is very small but still exists.

Just as a matter of reference you can lookup who owns these octets at the following link

http://standards.ieee.org/regauth/oui/index.shtml

A listing of registered organization can be found at the following link

http://standards.ieee.org/regauth/oui/oui.txt

In theory we should be able to use the private octets from the second listing in the macgen.py script to minimize any chances of a mac address conflict.

 

 

Great. I had been meaning to

Great. I had been meaning to look up more on MAC addresses wrt VMs.

BTW, there's another versatile MAC generator script at:
http://www.easyvmx.com/downloads.shtml

Thanks.

Re: easyvmx script

Good find.

This script can produce mac addresses with oui prefixes that various organizations own, namely vmware and xensource. Also produces randomly generated mac addresses, static mac addresses etc. Handy script.

Bridge Networking with Wireless nic with parprouted

explanations for virtualbox : http://www.hazard.maks.net/blog/index.php?op=Default&Date=20070120&blogId=1

then just need to start kvm virtual machine like this :
/usr/local/kvm/bin/qemu-system-x86_64 -hda /dev/sdb -m 512 -net nic,macaddr=DE:AD:BE:EF:90:26 -net tap,ifname=tap0,script=no

Last command not working for me

i have done all steps successfully but at last found above error, Any Idea ?

[root@localhost ~]# /usr/bin/qemu-system-x86_64 -hda /dev/lvmdata/qemuvm.root -m 512 -net nic,macaddr=00:16:3e:75:09:aa -net tap,script=/etc/qemu-ifup
/etc/qemu-ifup: could not launch network script
Could not initialize device 'tap'

Re: Last command not working for me

got the same problem

made the file executable (chmod +x ...) and everything worked :D

Post new comment

The content of this field is kept private and will not be shown publicly.