From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?ISO-8859-1?Q?Lu=EDs?= Silva Subject: Re: Re: [Xen-devel] ARP problems with xen 4.0 with pvops kernel 2.6.32.15 Date: Sun, 06 Jun 2010 20:14:46 +0100 Message-ID: <1275851686.2466.6.camel@luis-port> References: <294379.58011.qm@web56108.mail.re3.yahoo.com> Reply-To: luis.silva@axiomasoft.pt Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1773453433==" Return-path: In-Reply-To: <294379.58011.qm@web56108.mail.re3.yahoo.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-users-bounces@lists.xensource.com Errors-To: xen-users-bounces@lists.xensource.com To: Boris Derzhavets Cc: Jeremy Fitzhardinge , xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --===============1773453433== Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-VdKuLwYdVD+zhoq2STAW" --=-VdKuLwYdVD+zhoq2STAW Content-Type: multipart/alternative; boundary="=-frMvGYPkvxhzTc6koPVH" --=-frMvGYPkvxhzTc6koPVH Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello, Maybe you are having the same problem as I am. If you install the vm as an HVM you should be able to complete the installation. After that, whenever you want network working maintain a ping running on the domU side. As domU doesn't get the arp requests, even when communicating with the outside it doesn't always work. I'm no networking specialist, but I guess it should be because of arp cache clearing... Lu=C3=ADs On Sun, 2010-06-06 at 11:54 -0700, Boris Derzhavets wrote: > Virt-install hangs attempting to retrieve updates.img from Apache > Mirror setup at Dom0 > with kernel 2.6.32.15 > It doesn't happen with 2.6.32.10 (12,14 final). > Environment Xen 4.0 Dom0 on top Ubuntu 10.04 Server. Libvirt is > 0.8.0 . > Attempting to virt-install F13 PV DomU in vnc mode. >=20 > I believe that builds are consistent. > Runtime behavior is the same for .10, .14, .15. Seems to be > communicating problem between DomU and Dom0 during HTTP download. >=20 > Boris. >=20 > --- On Sun, 6/6/10, Jeremy Fitzhardinge wrote: >=20 > =20 > From: Jeremy Fitzhardinge > Subject: Re: [Xen-users] Re: [Xen-devel] ARP problems with xen > 4.0 with pvops kernel 2.6.32.15 > To: "Boris Derzhavets" > Cc: luis.silva@axiomasoft.pt, xen-devel@lists.xensource.com, > xen-users@lists.xensource.com > Date: Sunday, June 6, 2010, 12:43 PM > =20 > =20 > On 06/06/2010 03:19 AM, Boris Derzhavets wrote: > > Network issues when working with DomUs in 2.6.32.14 and > finally been > > fixed, > > seem to appear again in 2.6.32.15. Reverting to back to > xen/stable - > > 2.6.32.10 > > works as a fix again. > > > =20 > There are no substantial differences between 2.6.32.14 > and .15. If > there are any differences in behaviour between them, then I'd > suspect > some inconsistency from boot to boot, or in your kernel build > process. > =20 > J > =20 > > > > Boris > > > > --- On *Thu, 6/3/10, Lu=C3=ADs > Silva //* wrote: > > > > > > From: Lu=C3=ADs Silva > > Subject: Re: [Xen-users] Re: [Xen-devel] ARP problems > with xen 4.0 > > with pvops kernel > > To: "Boris Derzhavets" > > Cc: "Jeremy Fitzhardinge" , > > xen-devel@lists.xensource.com, > xen-users@lists.xensource.com > > Date: Thursday, June 3, 2010, 6:20 AM > > > > Hello, > > > > Thanks for the suggestion, xen/stable works ok for me. > Only > > problem is that I have to disable offload do get dhcp to > work on > > domU, but the problem I described before doesn't exist > in this > > kernel. Later today I'm going to try a previous build I > have based > > on stable-2.6.32.x (2.6.32.13) to check if it already > had this > > problem or not and I'll post the results. > > > > Lu=C3=ADs > > > > On Wed, 2010-06-02 at 12:26 -0700, Boris Derzhavets > wrote: > >> Could you,please, build and try 2.6.32.10 > ( xen/stable) ? > >> > >> Boris. > >> > >> --- On *Wed, 6/2/10, Lu=C3=ADs Silva > **//* > >> wrote: > >> > >> > >> From: Lu=C3=ADs Silva > >> Subject: [Xen-users] Re: [Xen-devel] ARP problems > with xen > >> 4.0 with pvops kernel > >> To: "Jeremy Fitzhardinge" > >> Cc: xen-devel@lists.xensource.com, > xen-users@lists.xensource.com > >> Date: Wednesday, June 2, 2010, 2:53 PM > >> > >> Hello, > >> > >> On Wed, 2010-06-02 at 09:06 -0700, Jeremy > Fitzhardinge wrote: > >>> On 06/02/2010 01:47 AM, Lu=C3=ADs Silva wrote: > >>> > Hello, > >>> > > >>> > I'm using the latest stable-2.6.32.x. I already > tried "ethtool -K > >>> > tx off", but that didn't make any > difference. Also, this only > >>> > happen with pv, in hvm mode all works ok and the > domU sees the arp > >>> > messages... > >>> > >>> Yes, ARP is a new twist on network problems. I'm > guessing you're using > >>> hvm without stubdoms, which means that its > networking originates from > >>> qemu within dom0, whereas PV and HVM+stubdom comes > via netback. > >>> > >>> =20 > >> Yes, when I mentioned hvm I was talking about hvm > without > >> stubdoms. I haven't tried those yet. > >>> But aside from that, I'm stumped. Are you running > any firewalls on > >>> either side? Can you try disabling all the > offloads (tx, rx, gso, tso) > >>> on all the relevent interfaces (bridge, netback, > within the guest) and > >>> see if that changes anything? > >>> > >>> J > >>> > >>> =20 > >> > >> Ok, this is the bridge interface: > >> > >> brctl show > >> bridge name bridge id STP enabled > interfaces > >> virbr0 8000.feffffffffff no vif1.0 > >> > >> ifconfig virbr0 > >> virbr0 Link encap:Ethernet HWaddr > c2:ef:67:2b:a4:23 =20 > >> inet addr:192.168.120.254 > Bcast:192.168.120.255 Mask:255.255.255.0 > >> inet6 addr: fe80::c0ef:67ff:fe2b:a423/64 > Scope:Link > >> UP BROADCAST RUNNING MULTICAST MTU:1500 > Metric:1 > >> RX packets:0 errors:0 dropped:0 > overruns:0 frame:0 > >> TX packets:25 errors:0 dropped:0 > overruns:0 carrier:0 > >> collisions:0 txqueuelen:0=20 > >> RX bytes:0 (0.0 > >> B)=20 > >> TX bytes:4662 (4.6 KB) > >> =20 > >> > >> > >> I'm not using firewall other than the rules defined > by > >> libvirt. DomU has no firewall and the rules in dom0 > are only > >> these (virbr0 is natted to the outside, virbr1 is > routed. The > >> result is the same in either one of them): > >> > >> sudo iptables -L -n -v > >> Chain INPUT (policy ACCEPT 241K packets, 53M bytes) > >> pkts bytes target prot opt in out > source destination =20 > >> 0 0 ACCEPT udp -- virbr1 * > 0.0.0.0/0 0.0.0.0/0 udp dpt:53=20 > >> 0 0 ACCEPT tcp -- virbr1 * > 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 > >> > >> =20 > >> 0 0 ACCEPT udp -- virbr1 * > 0.0.0.0/0 0.0.0.0/0 udp dpt:67=20 > >> 0 0 ACCEPT tcp -- virbr1 * > 0.0.0.0/0 0.0.0.0/0 tcp dpt:67=20 > >> 8 515 ACCEPT udp -- virbr0 * > 0.0.0.0/0 0.0.0.0/0 udp dpt:53=20 > >> 0 0 > >> > >> ACCEPT tcp -- virbr0 * 0.0.0.0/0 > 0.0.0.0/0 tcp dpt:53=20 > >> 0 0 ACCEPT udp -- virbr0 * > 0.0.0.0/0 0.0.0.0/0 udp dpt:67=20 > >> 0 0 ACCEPT tcp -- virbr0 * > 0.0.0.0/0 0.0.0.0/0 tcp dpt:67=20 > >> > >> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) > >> pkts bytes target =20 > >> prot > >> opt in out source > destination =20 > >> 0 0 ACCEPT all -- * virbr1 > 0.0.0.0/0 192.168.121.0/24 =20 > >> 0 0 ACCEPT all -- virbr1 * > 192.168.121.0/24 0.0.0.0/0 =20 > >> 0 0 ACCEPT all -- virbr1 virbr1 > 0.0.0.0/0 =20 > >> > >> 0.0.0.0/0 =20 > >> 0 0 REJECT all -- * virbr1 > 0.0.0.0/0 0.0.0.0/0 reject-with > icmp-port-unreachable=20 > >> 0 0 REJECT all -- virbr1 * > 0.0.0.0/0 0.0.0.0/0 reject-with > icmp-port-unreachable=20 > >> 13 3448 ACCEPT all -- * virbr0 > 0.0.0.0/0 192.168.120.0/24 =20 > >> state > >> RELATED,ESTABLISHED=20 > >> 16 1374 ACCEPT all -- virbr0 * > 192.168.120.0/24 0.0.0.0/0 =20 > >> 0 0 ACCEPT all -- virbr0 virbr0 > 0.0.0.0/0 0.0.0.0/0 =20 > >> 0 0 REJECT all -- * virbr0 > 0.0.0.0/0 0.0.0.0/0 reject-with > icmp-port-unreachable=20 > >> 0 0 REJECT all --=20 > >> virbr0 > >> * 0.0.0.0/0 0.0.0.0/0 > reject-with icmp-port-unreachable=20 > >> > >> Chain OUTPUT (policy ACCEPT 233K packets, 27M > bytes) > >> pkts bytes target prot opt in out > source destination > >> =20 > >> > >> > >> > >> And these are the various offload parameters as set > at boot: > >> > >> Offload parameters for virbr0: > >> rx-checksumming: on > >> tx-checksumming: on > >> scatter-gather: on > >> tcp-segmentation-offload: on > >> udp-fragmentation-offload: on > >> generic-segmentation-offload: on > >> generic-receive-offload: off > >> large-receive-offload: off > >> > >> Offload parameters for vif1.0: > >> rx-checksumming: on > >> tx-checksumming: on > >> scatter-gather: on > >> tcp-segmentation-offload: on > >> udp-fragmentation-offload: off > >> generic-segmentation-offload: on > >> generic-receive-offload: off > >> large-receive-offload: off > >> > >> Offload parameters for eth0: > >> rx-checksumming: on > >> tx-checksumming: on > >> scatter-gather: on > >> tcp-segmentation-offload: on > >> udp-fragmentation-offload: off > >> generic-segmentation-offload: off > >> generic-receive-offload: off > >> large-receive-offload: off > >> =20 > >> > >> > >> To disable all checksuming I run the following > commands: > >> dom0: > >> > >> sudo ethtool -K virbr0 tx off sg off tso off gso > off gro off > >> sudo ethtool -K vif1.0 tx off sg off tso off gso > off gro off > >> =20 > >> > >> domU > >> > >> sudo ethtool -K eth0 tx off sg off tso off gso off > gro off > >> =20 > >> > >> > >> This managed to get all parameter to off in the > mentioned > >> interfaces, but unfortunately the result is the > same. The arp > >> requests get to vif1.0, but not to eth0 on the > domU. > >> > >> sudo tcpdump -i vif1.0 -n -vv arp > >> tcpdump: WARNING: vif1.0: no IPv4 address assigned > >> tcpdump: listening on vif1.0, link-type EN10MB > (Ethernet), capture size 96 bytes > >> 19:43:51.233378 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> 19:43:52.233164 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> 19:43:53.233166 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> 19:43:54.684214 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> 19:43:55.684218 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> 19:43:56.684232 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >> =20 > >> > >> > >> I hope this information is enough. If I can provide > anything > >> else to help debug or test, please just ask! ;) > >> > >> Thanks in advance, > >> Lu=C3=ADs > >> > >>> > > >>> > Thanks, > >>> > Lu=C3=ADs > >>> > > >>> > On Tue, 2010-06-01 at 18:20 -0700, Jeremy > Fitzhardinge wrote: > >>> >> On 06/01/2010 05:38 PM, Lu=C3=ADs Silva wrote: > >>> >> > Hello, > >>> >> > > >>> >> > Finally I managed to get a xen 4.0 working on > ubuntu 10.04 with pvops > >>> >> > kernel and libvirt. However I am having some > problems with > >>> >> > networking... after initial installation with > netinstall image in hvm > >>> >> > mode, when I transform the vm in xen pv (via > pygrub with the current > >>> >> > ubuntu kernel), networking startEd to act > weird... > >>> >> > > >>> >> > Basically I'm not using a network script from > xen. I define a bridge > >>> >> > (manually or via libvirt, the result is the > same) and I use vif-bridge > >>> >> > to connect the vif to it. But now the weird > part comes: I can > >>> >> > communicate from domU to dom0, but not the > other way > >>> around, > >>> unless I > >>> >> > keep a ping running from domU to dom0... > That's right, weird... while > >>> >> > trying the ping from dom0 to domU, I used > tcpdump both on the bridge, > >>> >> > on the vif and on the eth0 in the domU. The > arp packets never get to > >>> >> > domU, but they appear both in the bridge and > the vif sniff's... > >>> >> > >>> >> What version of kernel are you using in dom0 > and domU? There was a > >>> >> netback bug which caused problems with > dom0<->domU communication, but it > >>> >> has been fixed for a while in 2.6.32 (but only > recently in .31). The > >>> >> workaround is to disable tx checksum offload on > your bridge (ethtool -K > >>> >> tx off). > >>> >> > >>> >> J > >>> >> > >>> >> > > >>> >> > Here is the bridge: > >>> >> > ifconfig virbr0 > >>> >> > virbr0 Link encap:Ethernet HWaddr > fe:ff:ff:ff:ff:ff =20 > >>> >> > > >>> =20 > >>> inet addr:192.168.120.254 Bcast:192.168.120.255 > Mask:255.255.255.0 > >>> >> > inet6 addr: > fe80::7cee:4bff:fe82:e63f/64 Scope:Link > >>> >> > UP BROADCAST RUNNING MULTICAST > MTU:1500 Metric:1 > >>> >> > RX packets:16 errors:0 dropped:0 > overruns:0 frame:0 > >>> >> > TX packets:226 errors:0 dropped:0 > overruns:0 carrier:0 > >>> >> > collisions:0 txqueuelen:0=20 > >>> >> > RX bytes:952 (952.0 B) TX > bytes:13953 (13.9 KB) > >>> >> > > >>> >> > > >>> >> > brctl show > >>> >> > bridge name bridge id STP enabled > interfaces > >>> >> > virbr0 8000.feffffffffff no > vif5.0 > >>> >> > > >>> >> > > >>> >> > tcpdump -i virbr0 -vv -n > >>> >> > tcpdump: listening on virbr0, link-type > EN10MB (Ethernet), capture size 96 bytes > >>> >> > 01:31:25.945151 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], > >>> proto ICMP (1), > >>> length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 1, length 64 > >>> >> > 01:31:26.945361 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 2, length 64 > >>> >> > 01:31:27.945420 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 3, length 64 > >>> >> > 01:31:28.945362 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 4, length 64 > >>> >> > 01:31:29.945364 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, > >>> seq 5, length > >>> 64 > >>> >> > 01:31:30.944300 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:30.945359 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 6, length 64 > >>> >> > 01:31:31.944297 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:31.945444 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> > 192.168.120.254 > 192.168.120.1: ICMP > echo request, id 10317, seq 7, length 64 > >>> >> > 01:31:32.944294 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:32.945401 IP (tos 0x0, ttl 64, id 0, > offset 0, flags [DF], proto ICMP (1), length 84) > >>> >> >=20 > >>> =20 > >>> 192.168.120.254 > 192.168.120.1: ICMP echo > request, id 10317, seq 8, length 64 > >>> >> > 01:31:33.947293 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:34.947373 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:35.947353 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:37.948352 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:38.948399 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:39.948376 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:31:40.949356 ARP, Ethernet (len 6), IPv4 > (len 4), Request > >>> who-has > >>> 192.168.120.1 tell 192.168.120.254, length 28 > >>> >> > > >>> >> > > >>> >> > tcpdump -i vif5.0 -vv -n > >>> >> > tcpdump: WARNING: vif5.0: no IPv4 address > assigned > >>> >> > tcpdump: listening on vif5.0, link-type > EN10MB (Ethernet), capture size 96 bytes > >>> >> > 01:32:19.956358 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:20.956358 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:21.956359 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:23.957311 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:24.957312 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length > >>> 28 > >>> >> > > >>> 01:32:25.957359 ARP, Ethernet (len 6), IPv4 (len > 4), Request who-has 192.168.120.1 tell 192.168.120.254, length > 28 > >>> >> > 01:32:27.958360 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:28.958310 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > 01:32:29.958362 ARP, Ethernet (len 6), IPv4 > (len 4), Request who-has 192.168.120.1 tell 192.168.120.254, > length 28 > >>> >> > =20 > >>> >> > > >>> >> > > >>> >> > Forwarding and iptables don't seem to be the > problem, because if I > >>> >> > initiate a ping from domU (at the same time > as the failing one from > >>> >> > dom0), the ping in dom0 starts to work. As > soon as I stop the ping in > >>> >> > domU, the one in dom0 starts failing again... > >>> >> > > >>> >> > Is anyone having the same > >>> problem? Is this a bug > >>> in the kernel? In > >>> >> > dom0 or domU? > >>> >> > > >>> >> > Thanks in advance, > >>> >> > Lu=C3=ADs > >>> >> > > >>> >> > > >>> >> > > _______________________________________________ > >>> >> > Xen-devel mailing list > >>> >> > Xen-devel@lists.xensource.com > > > >>> >> > http://lists.xensource.com/xen-devel > >>> >> > =20 > >>> >> > >>> >> > >>> >> _______________________________________________ > >>> >> Xen-devel mailing list > >>> >> Xen-devel@lists.xensource.com > > > >>> >> http://lists.xensource.com/xen-devel > >>> >> > >>> >> =20 > >>> > > >>> > >>> > >>> =20 > >> > >> > >> > >> -----Inline Attachment Follows----- > >> > >> _______________________________________________ > >> Xen-users mailing list > >> Xen-users@lists.xensource.com > >> http://lists.xensource.com/xen-users=20 > >> > >> > > > > > > -----Inline Attachment Follows----- > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > > > http://lists.xensource.com/xen-users > > > > > =20 > =20 >=20 --=-frMvGYPkvxhzTc6koPVH Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello,

Maybe you are having the same problem as I am. If you install the vm as an = HVM you should be able to complete the installation. After that, whenever y= ou want network working maintain a ping running on the domU side. As domU d= oesn't get the arp requests, even when communicating with the outside it do= esn't always work. I'm no networking specialist, but I guess it should be b= ecause of arp cache clearing...

Luís

On Sun, 2010-06-06 at 11:54 -0700, Boris Derzhavets wrote:
Virt-install hangs attempting to retrieve updates.img from Apache Mirror se= tup at Dom0
with kernel 2.6.32.15
It doesn't happen with 2.6.32.10 (12,14 final).
Environment Xen 4.0 Dom0 on top Ubuntu 10.04 Server. Libvirt is 0.8.0 .
Attempting  to virt-install F13 PV DomU in vnc mode.

  I believe that builds are consistent.
Runtime behavior is the same for .10, .14, .15. Seems to be communicating p= roblem between DomU and Dom0 during HTTP download.

Boris.

--- On Sun, 6/6/10, Jeremy Fitzhardinge <jeremy@goop.org>= ; wrote:

From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-users] Re: [Xen-devel] ARP problems with xen 4.0 with= pvops kernel 2.6.32.15
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: luis.silva@axiomasoft.pt, xen-devel@lists.xensource.com, xen-users@= lists.xensource.com
Date: Sunday, June 6, 2010, 12:43 PM

On 06/06/2010 03:19 AM, Boris Derzhavets wrote:
> Network issues when working with DomUs in 2.6.32.14 and finally be= en
> fixed,
> seem to appear again in 2.6.32.15. Reverting to back to xen/stable= -
> 2.6.32.10
> works as a fix again.
>

There are no substantial differences between 2.6.32.14 and .15.  I= f
there are any differences in behaviour between them, then I'd suspect some inconsistency from boot to boot, or in your kernel build process.<= BR>
    J

>
> Boris
>
> --- On *Thu, 6/3/10, Luís Silva /<luis.silva@axiomasoft.pt>/* wrote:
>
>
>     From: Luís Silva <luis.silva@axiomasoft.pt>
>     Subject: Re: [Xen-users] Re: [Xen-devel] A= RP problems with xen 4.0
>     with pvops kernel
>     To: "Boris Derzhavets" <bderzhavets@yahoo.com>=
>     Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
>     xen-devel@lists.xensource.com, xen-users@lists.xensource.com
>     Date: Thursday, June 3, 2010, 6:20 AM
>
>     Hello,
>
>     Thanks for the suggestion, xen/stable work= s ok for me. Only
>     problem is that I have to disable offload = do get dhcp to work on
>     domU, but the problem I described before d= oesn't exist in this
>     kernel. Later today I'm going to try a pre= vious build I have based
>     on stable-2.6.32.x (2.6.32.13) to check if= it already had this
>     problem or not and I'll post the results.<= BR> >
>     Luís
>
>     On Wed, 2010-06-02 at 12:26 -0700, Boris D= erzhavets wrote:
>>     Could you,please, build and try 2.6.32= .10 ( xen/stable) ?
>>
>>     Boris.
>>
>>     --- On *Wed, 6/2/10, Luís Silva *= */<luis.silva@axio= masoft.pt>/*
>>     wrote:
>>
>>
>>         From: Luís Silva &l= t;luis.silva@axiomaso= ft.pt>
>>         Subject: [Xen-users] Re:= [Xen-devel] ARP problems with xen
>>         4.0 with pvops kernel >>         To: "Jeremy Fitzhar= dinge" <jeremy@goop.or= g>
>>         Cc: xen-devel@lists.xensource.com, <= A HREF=3D"/mc/compose?to=3Dxen-users@lists.xensource.com">xen-users@lists.x= ensource.com
>>         Date: Wednesday, June 2,= 2010, 2:53 PM
>>
>>         Hello,
>>
>>         On Wed, 2010-06-02 at 09= :06 -0700, Jeremy Fitzhardinge wrote:
>>>         On 06/02/2010 01:47 = AM, Luís Silva wrote:
>>>         > Hello,
>>>         >
>>>         > I'm using the l= atest stable-2.6.32.x. I already tried "ethtool -K
>>>         > <bridge> = tx off", but that didn't make any difference. Also, this only
>>>         > happen with pv,= in hvm mode all works ok and the domU sees the arp
>>>         > messages...
>>>
>>>         Yes, ARP is a new tw= ist on network problems.  I'm guessing you're using
>>>         hvm without stubdoms= , which means that its networking originates from
>>>         qemu within dom0, wh= ereas PV and HVM+stubdom comes via netback.
>>>
>>>                &nb= sp;              
>>         Yes, when I mentioned hv= m I was talking about hvm without
>>         stubdoms. I haven't trie= d those yet.
>>>         But aside from that,= I'm stumped.  Are you running any firewalls on
>>>         either side? &n= bsp; Can you try disabling all the offloads (tx, rx, gso, tso)
>>>         on all the relevent = interfaces (bridge, netback, within the guest) and
>>>         see if that changes = anything?
>>>
>>>             J
>>>
>>>                &nb= sp;              
>>
>>         Ok, this is the bridge i= nterface:
>>
>>         brctl show
>>         bridge name  &= nbsp; bridge id        STP enabled  = ;  interfaces
>>         virbr0   =     8000.feffffffffff    no   = ;     vif1.0
>>
>>         ifconfig virbr0
>>         virbr0    Link= encap:Ethernet  HWaddr c2:ef:67:2b:a4:23 
>>                 &= nbsp; inet addr:192.168.120.254  Bcast:192.168.120.255  Mask= :255.255.255.0
>>                 &= nbsp; inet6 addr: fe80::c0ef:67ff:fe2b:a423/64 Scope:Link
>>                 &= nbsp; UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>                 &= nbsp; RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>                 &= nbsp; TX packets:25 errors:0 dropped:0 overruns:0 carrier:0
>>                 &= nbsp; collisions:0 txqueuelen:0
>>                 &= nbsp; RX bytes:0 (0.0
>>          B)
>>          TX bytes:4662 (4.6 KB)
>>                  =            
>>
>>
>>         I'm not using firewall o= ther than the rules defined by
>>         libvirt. DomU has no fir= ewall and the rules in dom0 are only
>>         these (virbr0 is natted = to the outside, virbr1 is routed. The
>>         result is the same in ei= ther one of them):
>>
>>         sudo iptables -L -n -v >>         Chain INPUT (policy ACCE= PT 241K packets, 53M bytes)
>>          pkts bytes target  &nbs= p;  prot opt in     out    &nb= sp;source               destin= ation         
>>             0  &n= bsp;  0 ACCEPT     udp  --  virbr1 = *       0.0.0.0/0         = ;   0.0.0.0/0           udp dpt:53 =
>>             0  &n= bsp;  0 ACCEPT     tcp  --  virbr1 = *       0.0.0.0/0         = ;   0.0.0.0/0           tcp dpt:53<= BR> >>
>>         
>>             0  &n= bsp;  0 ACCEPT     udp  --  virbr1 = *       0.0.0.0/0         = ;   0.0.0.0/0           udp dpt:67 =
>>             0  &n= bsp;  0 ACCEPT     tcp  --  virbr1 = *       0.0.0.0/0         = ;   0.0.0.0/0           tcp dpt:67 =
>>             8 &nb= sp; 515 ACCEPT     udp  --  virbr0 *&nbs= p;      0.0.0.0/0          &nb= sp; 0.0.0.0/0           udp dpt:53
>>             0  &n= bsp;  0
>>
>>          ACCEPT    &nb= sp;tcp  --  virbr0 *       0.0.0.0/0&nbs= p;           0.0.0.0/0        =    tcp dpt:53
>>             0  &n= bsp;  0 ACCEPT     udp  --  virbr0 = *       0.0.0.0/0         = ;   0.0.0.0/0           udp dpt:67 =
>>             0  &n= bsp;  0 ACCEPT     tcp  --  virbr0 = *       0.0.0.0/0         = ;   0.0.0.0/0           tcp dpt:67 =
>>
>>         Chain FORWARD (policy AC= CEPT 0 packets, 0 bytes)
>>          pkts bytes target  &nbs= p;
>>          prot
>>          opt in    &nb= sp;out     source          &nb= sp;    destination          >>             0  &n= bsp;  0 ACCEPT     all  --  * =     virbr1  0.0.0.0/0          &nb= sp; 192.168.121.0/24   
>>             0  &n= bsp;  0 ACCEPT     all  --  virbr1 = *       192.168.121.0/24     0= .0.0.0/0           
>>             0  &n= bsp;  0 ACCEPT     all  --  virbr1 = virbr1  0.0.0.0/0           
>>
>>          0.0.0.0/0     = ;      
>>             0  &n= bsp;  0 REJECT     all  --  * =     virbr1  0.0.0.0/0          &nb= sp; 0.0.0.0/0           reject-with icmp= -port-unreachable
>>             0  &n= bsp;  0 REJECT     all  --  virbr1 = *       0.0.0.0/0         = ;   0.0.0.0/0           reject-with= icmp-port-unreachable
>>            13  3448 ACCEPT&= nbsp;    all  --  *      virbr0&nbs= p; 0.0.0.0/0            192.168.120.0/24 = ;  
>>          state
>>          RELATED,ESTABLISHED
>>            16  1374 ACCEPT&= nbsp;    all  --  virbr0 *     &nbs= p; 192.168.120.0/24     0.0.0.0/0    &nb= sp;      
>>             0  &n= bsp;  0 ACCEPT     all  --  virbr0 = virbr0  0.0.0.0/0            0.0.0.0/0&n= bsp;          
>>             0  &n= bsp;  0 REJECT     all  --  * =     virbr0  0.0.0.0/0          &nb= sp; 0.0.0.0/0           reject-with icmp= -port-unreachable
>>             0  &n= bsp;  0 REJECT     all  --
>>          virbr0
>>          *      &= nbsp;0.0.0.0/0            0.0.0.0/0  &nb= sp;        reject-with icmp-port-unreachable
>>
>>         Chain OUTPUT (policy ACC= EPT 233K packets, 27M bytes)
>>          pkts bytes target  &nbs= p;  prot opt in     out    &nb= sp;source               destin= ation
>>                  =            
>>
>>
>>
>>         And these are the variou= s offload parameters as set at boot:
>>
>>         Offload parameters for v= irbr0:
>>         rx-checksumming: on
>>         tx-checksumming: on
>>         scatter-gather: on
>>         tcp-segmentation-offload= : on
>>         udp-fragmentation-offloa= d: on
>>         generic-segmentation-off= load: on
>>         generic-receive-offload:= off
>>         large-receive-offload: o= ff
>>
>>         Offload parameters for v= if1.0:
>>         rx-checksumming: on
>>         tx-checksumming: on
>>         scatter-gather: on
>>         tcp-segmentation-offload= : on
>>         udp-fragmentation-offloa= d: off
>>         generic-segmentation-off= load: on
>>         generic-receive-offload:= off
>>         large-receive-offload: o= ff
>>
>>         Offload parameters for e= th0:
>>         rx-checksumming: on
>>         tx-checksumming: on
>>         scatter-gather: on
>>         tcp-segmentation-offload= : on
>>         udp-fragmentation-offloa= d: off
>>         generic-segmentation-off= load: off
>>         generic-receive-offload:= off
>>         large-receive-offload: o= ff
>>                  =            
>>
>>
>>         To disable all checksumi= ng I run the following commands:
>>         dom0:
>>
>>         sudo ethtool -K virbr0 t= x off sg off tso off gso off gro off
>>         sudo ethtool -K vif1.0 t= x off sg off tso off gso off gro off
>>                  =            
>>
>>         domU
>>
>>         sudo ethtool -K eth0 tx = off sg off tso off gso off gro off
>>                  =            
>>
>>
>>         This managed to get all = parameter to off in the mentioned
>>         interfaces, but unfortun= ately the result is the same. The arp
>>         requests get to vif1.0, = but not to eth0 on the domU.
>>
>>         sudo tcpdump -i vif1.0 -= n -vv arp
>>         tcpdump: WARNING: vif1.0= : no IPv4 address assigned
>>         tcpdump: listening on vi= f1.0, link-type EN10MB (Ethernet), capture size 96 bytes
>>         19:43:51.233378 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>         19:43:52.233164 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>         19:43:53.233166 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>         19:43:54.684214 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>         19:43:55.684218 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>         19:43:56.684232 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>                  =            
>>
>>
>>         I hope this information = is enough. If I can provide anything
>>         else to help debug or te= st, please just ask! ;)
>>
>>         Thanks in advance,
>>         Luís
>>
>>>         >
>>>         > Thanks,
>>>         > Luís
>>>         >
>>>         > On Tue, 2010-06= -01 at 18:20 -0700, Jeremy Fitzhardinge wrote:
>>>         >> On 06/01/20= 10 05:38 PM, Luís Silva wrote:
>>>         >> > Hello,=
>>>         >> >
>>>         >> > Finall= y I managed to get a xen 4.0 working on ubuntu 10.04 with pvops
>>>         >> > kernel= and libvirt. However I am having some problems with
>>>         >> > networ= king... after initial installation with netinstall image in hvm
>>>         >> > mode, = when I transform the vm in xen pv (via pygrub with the current
>>>         >> > ubuntu= kernel), networking startEd to act weird...
>>>         >> >
>>>         >> > Basica= lly I'm not using a network script from xen. I define a bridge
>>>         >> > (manua= lly or via libvirt, the result is the same) and I use vif-bridge
>>>         >> > to con= nect the vif to it. But now the weird part comes: I can
>>>         >> > commun= icate from domU to dom0, but not the other way
>>>          around,
>>>          unless I
>>>         >> > keep a= ping running from domU to dom0... That's right, weird... while
>>>         >> > trying= the ping from dom0 to domU, I used tcpdump both on the bridge,
>>>         >> > on the= vif and on the eth0 in the domU. The arp packets never get to
>>>         >> > domU, = but they appear both in the bridge and the vif sniff's...
>>>         >>
>>>         >> What versio= n of kernel are you using in dom0 and domU?  There was a
>>>         >> netback bug= which caused problems with dom0<->domU communication, but it
>>>         >> has been fi= xed for a while in 2.6.32 (but only recently in .31).  The
>>>         >> workaround = is to disable tx checksum offload on your bridge (ethtool -K
>>>         >> <bridge&= gt; tx off).
>>>         >>
>>>         >>   = ;  J
>>>         >>
>>>         >> >
>>>         >> > Here i= s the bridge:
>>>         >> > ifconf= ig virbr0
>>>         >> > virbr0=     Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff 
>>>         >> >
>>>                &nb= sp;  
>>>          inet addr:192.168.120.25= 4  Bcast:192.168.120.255  Mask:255.255.255.0
>>>         >> >  =          inet6 addr: fe80::7cee:4bff:fe82:e63= f/64 Scope:Link
>>>         >> >  =          UP BROADCAST RUNNING MULTICAST = MTU:1500  Metric:1
>>>         >> >  =          RX packets:16 errors:0 dropped:0 ove= rruns:0 frame:0
>>>         >> >  =          TX packets:226 errors:0 dropped:0 ov= erruns:0 carrier:0
>>>         >> >  =          collisions:0 txqueuelen:0
>>>         >> >  =          RX bytes:952 (952.0 B)  TX byte= s:13953 (13.9 KB)
>>>         >> >
>>>         >> >
>>>         >> > brctl = show
>>>         >> > bridge= name    bridge id        STP = enabled    interfaces
>>>         >> > virbr0=         8000.feffffffffff    n= o        vif5.0
>>>         >> >
>>>         >> >
>>>         >> > tcpdum= p -i virbr0 -vv -n
>>>         >> > tcpdum= p: listening on virbr0, link-type EN10MB (Ethernet), capture size 96 bytes<= BR> >>>         >> > 01:31:= 25.945151 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF],
>>>          proto ICMP (1),
>>>          length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 1, length 64
>>>         >> > 01:31:= 26.945361 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 2, length 64
>>>         >> > 01:31:= 27.945420 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 3, length 64
>>>         >> > 01:31:= 28.945362 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 4, length 64
>>>         >> > 01:31:= 29.945364 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317,
>>>          seq 5, length
>>>          64
>>>         >> > 01:31:= 30.944300 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 30.945359 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 6, length 64
>>>         >> > 01:31:= 31.944297 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 31.945444 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >  =    192.168.120.254 > 192.168.120.1: ICMP echo request, id= 10317, seq 7, length 64
>>>         >> > 01:31:= 32.944294 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 32.945401 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), = length 84)
>>>         >> >
>>>           
>>>          192.168.120.254 > 192= .168.120.1: ICMP echo request, id 10317, seq 8, length 64
>>>         >> > 01:31:= 33.947293 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 34.947373 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 35.947353 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 37.948352 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 38.948399 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 39.948376 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:31:= 40.949356 ARP, Ethernet (len 6), IPv4 (len 4), Request
>>>          who-has
>>>          192.168.120.1 tell 192.1= 68.120.254, length 28
>>>         >> >
>>>         >> >
>>>         >> > tcpdum= p -i vif5.0 -vv -n
>>>         >> > tcpdum= p: WARNING: vif5.0: no IPv4 address assigned
>>>         >> > tcpdum= p: listening on vif5.0, link-type EN10MB (Ethernet), capture size 96 bytes<= BR> >>>         >> > 01:32:= 19.956358 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 20.956358 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 21.956359 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 23.957311 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 24.957312 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length
>>>          28
>>>         >> >
>>>          01:32:25.957359 ARP, Eth= ernet (len 6), IPv4 (len 4), Request who-has 192.168.120.1 tell 192.168.120= .254, length 28
>>>         >> > 01:32:= 27.958360 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 28.958310 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > 01:32:= 29.958362 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.120.= 1 tell 192.168.120.254, length 28
>>>         >> > &= nbsp; 
>>>         >> >
>>>         >> >
>>>         >> > Forwar= ding and iptables don't seem to be the problem, because if I
>>>         >> > initia= te a ping from domU (at the same time as the failing one from
>>>         >> > dom0),= the ping in dom0 starts to work. As soon as I stop the ping in
>>>         >> > domU, = the one in dom0 starts failing again...
>>>         >> >
>>>         >> > Is any= one having the same
>>>          problem? Is this a bug >>>          in the kernel? In
>>>         >> > dom0 o= r domU?
>>>         >> >
>>>         >> > Thanks= in advance,
>>>         >> > Lu= 7;s
>>>         >> >
>>>         >> >
>>>         >> > ______= _________________________________________
>>>         >> > Xen-de= vel mailing list
>>>         >> > Xen-devel@lists.xensou= rce.com <mailto:Xen-devel@lists.xensource.com> <mailto:Xen-devel@lists.xensource.com&g= t;
>>>         >> > http://lists.xensource.com/xen-d= evel
>>>         >> > &= nbsp; 
>>>         >>
>>>         >>
>>>         >> ___________= ____________________________________
>>>         >> Xen-devel m= ailing list
>>>         >> Xen-devel@lists.xensource.c= om <mailto:Xen-devel@lists.xensource.com> <mailto:Xen-devel@lists.xensource.com> >>>         >> http://lists.xensource.com/xen-devel<= /A>
>>>         >>
>>>         >>   = ;  
>>>         >
>>>
>>>
>>>                &nb= sp;              
>>
>>
>>
>>         -----Inline Attachment F= ollows-----
>>
>>         ________________________= _______________________
>>         Xen-users mailing list >>         Xen-users@lists.xensource.com
>>         http://lists.xensource.com/xen-users
>>
>>
>
>
>     -----Inline Attachment Follows-----
>
>     __________________________________________= _____
>     Xen-users mailing list
>     Xen-users@lists.xensource.com
>     </mc/compose?to=3DXen-users@lists.xensource.com>=
>     http://lists.xensource.com/xen-users
>
>


--=-frMvGYPkvxhzTc6koPVH-- --=-VdKuLwYdVD+zhoq2STAW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAkwL85gACgkQXSU0BM8MHj8r3gCgjdSNHLdpFqK0Vsbu84d7s2K1 6lwAnikPhDvXnDz1RwSwsTuvXSvSskKg =I/2N -----END PGP SIGNATURE----- --=-VdKuLwYdVD+zhoq2STAW-- --===============1773453433== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users --===============1773453433==--