Step by step instructions for KVM networking

Step by step instructions for KVM networking

[Daniel Qarras]

Hi all,

I've played with Xen on my Fedora and it's been working fine. Now I'm
testing KVM with 2.6.20 kernel and it's working nicely, too, but I'm
unable to setup a full-blown networking, i.e., a mode where also
icmp/udp work. User-mode networking works well, e.g., yum and firefox
run just fine under KVM.

Could someone please post a step by step list how to enable
non-usermode networking with KVM/QEMU on Fedora? I've googled
for this and found instructions (mostly for Debian) but I've not been
able to set up this on Fedora.

(My test machine has one NIC with a 10.x.x.x address in an intranet. I
would want that my KVM guests have, e.g., 192.168.x.x static addresses
and they can ping each other, the host machine, and access outside
world with tcp/udp/icmp etc. So something what Xen does now
automatically on Fedora. Connection initiated from outside world are
not essential but would of course be a nice bonus.)

Thanks.

[I posted this earlier to a Fedora list but got no reply there.]

http://www.redhat.com/archives/fedora-test-list/2007-January/msg00405.html




 
____________________________________________________________________________________
Now that's room service!  Choose from over 150,000 hotels
in 45,000 destinations on Yahoo! Travel to find your fit.
http://farechase.yahoo.com/promo-generic-14795097

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: Step by step instructions for KVM networking

[Avi Kivity]

Daniel Qarras wrote:
> Hi all,
>
> I've played with Xen on my Fedora and it's been working fine. Now I'm
> testing KVM with 2.6.20 kernel and it's working nicely, too, but I'm
> unable to setup a full-blown networking, i.e., a mode where also
> icmp/udp work. User-mode networking works well, e.g., yum and firefox
> run just fine under KVM.
>
> Could someone please post a step by step list how to enable
> non-usermode networking with KVM/QEMU on Fedora? I've googled
> for this and found instructions (mostly for Debian) but I've not been
> able to set up this on Fedora.
>   

In general the setup is exactly like qemu, so any howto for qemu is 
applicable.

Here's what I do:

- copy scripts/kvm from the kvm tarball to /etc/init.d, chkconfig and 
start the kvm service. that sets up a bridge
- copy scripts/qemu-ifup into a new directory /etc/kvm.  make sure it is 
executable.
- run qemu like this:

   qemu-system-x86_64 -net nic -net tap,script=/etc/kvm/qemu-ifup [other 
options]...


> (My test machine has one NIC with a 10.x.x.x address in an intranet. I
> would want that my KVM guests have, e.g., 192.168.x.x static addresses
> and they can ping each other, the host machine, and access outside
> world with tcp/udp/icmp etc. So something what Xen does now
> automatically on Fedora. Connection initiated from outside world are
> not essential but would of course be a nice bonus.)
>   

The configuration above would put the guests on the same ethernet 
segment, so you'd need to assign them addresses in the 10.0.0.0/8 subnet.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: Step by step instructions for KVM networking

[Daniel Qarras]

Hi!

> In general the setup is exactly like qemu, so any howto for qemu is 
> applicable.
> 
> Here's what I do:
> 
> - copy scripts/kvm from the kvm tarball to /etc/init.d, chkconfig and
> start the kvm service. that sets up a bridge
> - copy scripts/qemu-ifup into a new directory /etc/kvm.  make sure it
> is executable.
> - run qemu like this:
> 
>    qemu-system-x86_64 -net nic -net tap,script=/etc/kvm/qemu-ifup
> [other options]...

Thanks for your kind answer but unfortunately I cannot find the scripts
you mentioned from the kvm tar ball:

sh-3.1$ find kvm-14 | grep scripts
kvm-14/scripts
kvm-14/scripts/run_img
kvm-14/scripts/65-kvm.rules
kvm-14/scripts/mkbootdisk
sh-3.1$ 

There is kvm-14/kvm script but that doesn't look at all like a Red Hat
init script, it a Python script. I googled for qemu-ifup and found
about dozen different kind of versions so I'm not sure which one I
should use. But definetely the scripts you mentioned would be just what
I was looking for!

Thanks.




 
____________________________________________________________________________________
Sucker-punch spam with award-winning protection. 
Try the free Yahoo! Mail Beta.
http://advision.webevents.yahoo.com/mailbeta/features_spam.html

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: Step by step instructions for KVM networking

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]

Daniel Qarras wrote:
> Hi!
>
>   
>> In general the setup is exactly like qemu, so any howto for qemu is 
>> applicable.
>>
>> Here's what I do:
>>
>> - copy scripts/kvm from the kvm tarball to /etc/init.d, chkconfig and
>> start the kvm service. that sets up a bridge
>> - copy scripts/qemu-ifup into a new directory /etc/kvm.  make sure it
>> is executable.
>> - run qemu like this:
>>
>>    qemu-system-x86_64 -net nic -net tap,script=/etc/kvm/qemu-ifup
>> [other options]...
>>     
>
> Thanks for your kind answer but unfortunately I cannot find the scripts
> you mentioned from the kvm tar ball:
>
> sh-3.1$ find kvm-14 | grep scripts
> kvm-14/scripts
> kvm-14/scripts/run_img
> kvm-14/scripts/65-kvm.rules
> kvm-14/scripts/mkbootdisk
> sh-3.1$ 
>
> There is kvm-14/kvm script but that doesn't look at all like a Red Hat
> init script, it a Python script. I googled for qemu-ifup and found
> about dozen different kind of versions so I'm not sure which one I
> should use. But definetely the scripts you mentioned would be just what
> I was looking for!
>
>   

Sorry, the scripts didn't make it into the release.  I'm attaching them 
here.

If you're running Fedora, it looks like there's some bridging support in 
initscripts (I haven't tried it though).


-- 
error compiling committee.c: too many arguments to function


[-- Attachment #2: kvm --]
[-- Type: text/plain, Size: 4510 bytes --]

#!/bin/sh
# kvm init script              Takes care for all VMM tasks
#
# chkconfig: - 99 01
# description: The KVM is a kernel level Virtual Machine Monitor.  \
#              Currently it starts a bridge and attached eth0 for it

dir=$(dirname "$0")

ifnum=${ifnum:-$(ip route list | awk '/^default / { print $NF }' | sed 's/^[^0-9]*//')}
ifnum=${ifnum:-0}
switch=${sw0:-sw${ifnum}}
pif=${pif:-eth${ifnum}}
antispoof=${antispoof:-no}
command=$1

if [ -f /etc/sysconfig/network-scripts/ifcfg-${pif} ]; then
    . /etc/sysconfig/network-scripts/ifcfg-${pif}
fi

get_ip_info() {
    addr=`ip addr show dev $1 | egrep '^ *inet' | sed -e 's/ *inet //' -e 's/ .*//'`
    gateway=$(ip route list | awk '/^default / { print $3 }')
}
    
do_ifup() {
     if [ ${addr} ] ; then
        ip addr flush $1 
        ip addr add ${addr} dev $1 
        ip link set dev $1 up 
     fi
}

link_exists()
{
    if ip link show "$1" >/dev/null 2>/dev/null
    then
        return 0
    else
        return 1
    fi
}

create_switch () {
    local switch=$1

    if [ ! -e "/sys/class/net/${switch}/bridge" ]; then
	brctl addbr ${switch} >/dev/null 2>&1
	brctl stp ${switch} off >/dev/null 2>&1
	brctl setfd ${switch} 0.1 >/dev/null 2>&1
    fi
    ip link set ${switch} up >/dev/null 2>&1
}

add_to_switch () {
    local switch=$1
    local dev=$2

    if [ -e "/sys/class/net/${switch}/brif/${dev}" ]; then
	ip link set ${dev} up >/dev/null 2>&1 || true
	return
    fi
    brctl addif ${switch} ${dev} >/dev/null 2>&1
    ip link set ${dev} up >/dev/null 2>&1
}

#taken from Xen
transfer_routes () {
    local src=$1
    local dst=$2
    # List all routes and grep the ones with $src in.
    # Stick 'ip route del' on the front to delete.
    # Change $src to $dst and use 'ip route add' to add.
    ip route list | sed -ne "
/dev ${src}\( \|$\)/ {
  h
  s/^/ip route del /
  P
  g
  s/${src}/${dst}/
  s/^/ip route add /
  P
  d
}" | sh -e
}


change_ips() {
    local src=$1
    local dst=$2

    #take care also for case we do not have /etc/sysconfig data (the switch as a src case)
    if [ -x $BOOTPROTO ]; then
        if [ -x $(pgrep dhclient) ];then
           BOOTPROTO="null"
        else
            BOOTPROTO="dhcp"
        fi
    fi

    if [ $BOOTPROTO = "dhcp" ]; then
        ifdown ${src} >/dev/null 2>&1 || true
        ip link set ${src} up >/dev/null 2>&1
        pkill dhclient >/dev/null 2>&1
	for ((i=0;i<3;i++)); do
	    pgrep dhclient >/dev/null 2>&1 || i=4	
   	    sleep 1
	done
        dhclient ${dst} >/dev/null 2>&1
    else
        get_ip_info ${src}
        ifdown ${src} >/dev/null 2>&1 || true
        ip link set ${src} up >/dev/null 2>&1
        do_ifup ${dst}
        transfer_routes ${src} ${dst}
        ip route add default via ${gateway} dev ${dst}
    fi
}
               
antispoofing () {
    iptables -P FORWARD DROP >/dev/null 2>&1
    iptables -F FORWARD >/dev/null 2>&1
    iptables -A FORWARD -m physdev --physdev-in ${dev} -j ACCEPT >/dev/null 2>&1
}

status () {
    local dev=$1
    local sw=$2
    
    echo '============================================================'
    ip addr show ${dev}
    ip addr show ${sw}
    echo ' '
    brctl show ${sw}
    echo ' '
    ip route list
    echo ' '
    route -n
    echo '============================================================'
    gateway=$(ip route list | awk '/^default / { print $3 }')
    ping -c 1 ${gateway} || true
    echo '============================================================'
}

start () {
    if [ "${switch}" = "null" ] ; then
	return
    fi

    create_switch ${switch}
    add_to_switch ${switch} ${pif}
    change_ips ${pif} ${switch}

    if [ ${antispoof} = 'yes' ] ; then
	antispoofing
    fi

    grep -q GenuineIntel /proc/cpuinfo && /sbin/modprobe kvm-intel
    grep -q AuthenticAMD /proc/cpuinfo && /sbin/modprobe kvm-amd
}

stop () {
    if [ "${switch}" = "null" ]; then
	return
    fi
    if ! link_exists "$switch"; then
	return
    fi

    change_ips ${switch} ${pif}
    ip link set ${switch} down
    brctl delbr ${switch}

    grep -q GenuineIntel /proc/cpuinfo && /sbin/modprobe -r kvm-intel
    grep -q AuthenticAMD /proc/cpuinfo && /sbin/modprobe -r kvm-amd
    /sbin/modprobe -r kvm
}


case "$command" in
    start)
	echo -n $"Starting KVM: "
	start
	echo
	;;
    
    stop)
	echo -n $"Shutting down KVM: "
	stop
	echo
	;;

    status)
	status ${pif} ${switch}
	;;

    *)
	echo "Unknown command: $command" >&2
	echo 'Valid commands are: start, stop, status' >&2
	exit 1
esac

[-- Attachment #3: qemu-ifup --]
[-- Type: text/plain, Size: 117 bytes --]

#!/bin/sh

switch=$(ip route list | awk '/^default / { print $NF }')
ifconfig $1 0.0.0.0 up
brctl addif ${switch} $1

[-- Attachment #4: Type: text/plain, Size: 345 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #5: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: Step by step instructions for KVM networking

[Avi Kivity]

Avi Kivity wrote:
>
> Sorry, the scripts didn't make it into the release.  I'm attaching 
> them here.
>
> If you're running Fedora, it looks like there's some bridging support 
> in initscripts (I haven't tried it though).
>

I tried it now, it's simple and it even works:

1. Add BRIDGE=sw0 to ifcfg-eth0
2. Create a new ifcfg-sw0 that has TYPE=Bridge, and any dhcp/ip 
configuration you previously had in ifcfg-eth0

For the Fedorans here, system-config-network doesn't seem to support 
it.  I'd say the initscripts support trumps any Xen/kvm specific setup, 
no?  All that's needed is GUI support and some script/button to convert 
an unbridged setup to a bridged one.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: Step by step instructions for KVM networking

[Daniel Qarras]

Hi Avi, all,

> I tried it now, it's simple and it even works:
> 
> 1. Add BRIDGE=sw0 to ifcfg-eth0
> 2. Create a new ifcfg-sw0 that has TYPE=Bridge, and any dhcp/ip 
> configuration you previously had in ifcfg-eth0

I've now managed to get several KVM hosts working in a local intranet
based on your help, thanks! Below are finally step-by-step instructions
that work at least on Fedora Core 6, running other FC6 virtual hosts:

- configure networking for virtual hosts:
  - eth0 used for outer/public network connections
  - eth1 used in a local intranet
  - cp ifcfg-eth1 ifcfg-sw1
  - add BRIDGE=sw1 to ifcfg-eth1
  - add TYPE=Bridge to ifcfg-sw1, change DEVICE=sw1
  - ONBOOT=yes probably useful on both, eth1 needs to be up first
  - restart network/boot and verify that eth0/eth1/sw1 are up, eth1 has
no IP
  - optional: enable NAT for KVM hosts:
    - echo 1 > /proc/sys/net/ipv4/ip_forward
    - iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    - make sure no other fw rules are blocking traffic to outer network
  - run dhcpd (if wanted) and name server on interface sw1. dnsmasq as
a name server is a very good choice, needs zero configuration. dhcpd
should provide addresses proper for the local intranet.
  - create executable /etc/qemu-ifup that contains:

#!/bin/sh
switch=sw1
ifconfig $1 0.0.0.0 up
brctl addif ${switch} $1

That's it! To run several KVM hosts, just do:

- download, compile, and install proper kvm version (v12 for 2.6.20,
v15 or newer for 2.6.21 and newer), see
http://kvm.qumranet.com/kvmwiki/HOWTO for details
- create a qcow disk image, see the above HOWTO
- launch FC6 installer (or other OS of your choice), use dhcp or static
addresses depending on your setup, see HOWTO for info to start install
- after installation start the OS (see HOWTO) and configure the virtual
host to the max
- shutdown the host, cp vdisk1.img vdisk2.img, cp vdisk1.img
vdisk3.img, etc as many times as needed
- finally, start virtual hosts with:

qemu-system-x86_64 -hda vdisk1.img -boot c -m 512 -net
nic,macaddr=00:00:00:00:00:11 -net tap,script=/etc/qemu-ifup &
qemu-system-x86_64 -hda vdisk2.img -boot c -m 512 -net
nic,macaddr=00:00:00:00:00:22 -net tap,script=/etc/qemu-ifup &

etc. Double check network settings on virtual hosts and they should be
all set to work as any other host on your local intranet: they can ping
each other, public hosts, you can log in to them, etc.

Cheers!


 
____________________________________________________________________________________
Never miss an email again!
Yahoo! Toolbar alerts you the instant new Mail arrives.
http://tools.search.yahoo.com/toolbar/features/mail/

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: Step by step instructions for KVM networking

[Steve Kwee]

Daniel Qarras <dqarras@...> writes:

> 
> Hi Avi, all,
> 
> > I tried it now, it's simple and it even works:
> > 
> > 1. Add BRIDGE=sw0 to ifcfg-eth0
> > 2. Create a new ifcfg-sw0 that has TYPE=Bridge, and any dhcp/ip 
> > configuration you previously had in ifcfg-eth0
> 
> I've now managed to get several KVM hosts working in a local intranet
> based on your help, thanks! Below are finally step-by-step instructions
> that work at least on Fedora Core 6, running other FC6 virtual hosts:
> 
:
:
Hi all,

some more hints that would have saved me a lot of time:

If your eth0 is a wireless device, it might be a good idea to switch to another
device to try this. Many wireless devices have problems with bridged networking.
I switched to my wired network after a lot of testing and had success.

If you do not provide a mac addr on the commandline of qemu / kvm a static
macaddr will be used, giving you problems starting with vm no2.
In the guest an interface eth0 is instantiated with this fixed macaddr
which can be configured as usual
If you provide a macaddr the interface is instantiated as eth1 instead of eth0.
I do not understand why, maybe someone can acknoledge this fact.
Anyway thinks worked when I switched to eth1 in my hosts.

An article that helped me a lot was:
http://kidsquid.com/cgi-bin/moin.cgi/QemuNetwork






-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV