Xen/AoE skb frag array incompatibility

Xen/AoE skb frag array incompatibility

[Tracy R Reed]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

Greetings, Xen developers!

I have been using AoE with Xen to great effect and I am very pleased 
with how the whole system has worked out. The two technologies really 
complement each other. I consider it the ultimate setup. Aside from the 
network switch (eventually to be remedied) my serving infrastructure has 
zero single points of failure.

However, this virtualization paradise has a flaw: There seems to be some 
sort of incompatibility in the way AoE uses the skb frag array in the 
latest version of AoE (6-40) and in the way Xen uses it. This causes new 
AoE block devices to fail to be accessible among potentially other problems.

Is anyone aware of this problem? It is not clear to me if it is a 
problem with Xen or if the fault lies in AoE. I have looked into it a 
bit but my kernel hacking talents are rather meager. Attached is a 
recent email from the AoE list discussing it. If you have any 
suggestions for the AoE folks they can be reached over at 
aoetools-discuss@lists.sourceforge.net

-- 
Tracy R Reed                  http://ultraviolet.org
A: Because we read from top to bottom, left to right
Q: Why should I start my reply below the quoted text

[-- Attachment #2: Re: [Aoetools-discuss] Unable to open dev /dev/etherd/e1.6: --]
[-- Type: message/rfc822, Size: 2309 bytes --]

From: Sam Hopkins <sah@coraid.com>
To: treed@ultraviolet.org, ecashin@coraid.com
Cc: aoetools-discuss@lists.sourceforge.net
Subject: Re: [Aoetools-discuss] Unable to open dev /dev/etherd/e1.6:
Date: Thu, 7 Dec 2006 10:29:25 -0500
Message-ID: <6bc653c54757da64842c5d286b15bee8@coraid.com>

> Ed L. Cashin wrote:
>> That sounds similar to the problem that Sam and Adi Kriegisch were
>> recently discussing.  It looked like an incompatibility between our
>> use of "zero-copy writes" and the network layer in his Xen kernels.
>> 
>> Adi Kriegisch said that version aoe6-23 worked for him.
> 
> Hmm...I tried to compile the aoe6-23 version and ran into the old error:
> 
> /root/aoe6-23/linux/drivers/block/aoe/aoenet.c:142: error: too many
> arguments to function `skb_linearize'
> 
> which I can easily fix...then I am reminded that with the aoe driver in
> the stock 2.6.16 kernel I would get the error "Can't schedule work for
> e1.1, it's already on! (This really shouldn't happen)" but it seems that
> is the very old version 14. Hopefully if I just fix the skb_linearize
> arguments in aoe6-23 it will work acceptably. I just hate being the only
>    one running a unique version of the driver.

It would be great if someone close to the Xen community would engage
them about our use of the skb frag array to figure out why this is
happening, since it's a problem that seems to be unique to that
environment.  It is a detriment to aoe that one must hack on old
drivers with missing features to overcome a bug that's already been
identified.

We're not going to maintain an aoe6-23 driver for Xen folks to
workaround a bug.  Fix the bug!

Cheers,

Sam


[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen/AoE skb frag array incompatibility

[Keir Fraser]

Is there a clear technical description of what the problem is? I see three
problems vaguely referred to in this email:
 1. Build failure on skb_linearize()
 2. Warnings about 'can't schedule work'
 3. Some unspecified problem with frag lists

If someone digs into the problem a bit and points us at some code we can
probably advise what should be done.

 -- Keir

On 8/12/06 2:00 am, "Tracy R Reed" <treed@ultraviolet.org> wrote:

> Greetings, Xen developers!
> 
> I have been using AoE with Xen to great effect and I am very pleased
> with how the whole system has worked out. The two technologies really
> complement each other. I consider it the ultimate setup. Aside from the
> network switch (eventually to be remedied) my serving infrastructure has
> zero single points of failure.
> 
> However, this virtualization paradise has a flaw: There seems to be some
> sort of incompatibility in the way AoE uses the skb frag array in the
> latest version of AoE (6-40) and in the way Xen uses it. This causes new
> AoE block devices to fail to be accessible among potentially other problems.
> 
> Is anyone aware of this problem? It is not clear to me if it is a
> problem with Xen or if the fault lies in AoE. I have looked into it a
> bit but my kernel hacking talents are rather meager. Attached is a
> recent email from the AoE list discussing it. If you have any
> suggestions for the AoE folks they can be reached over at
> aoetools-discuss@lists.sourceforge.net