Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

[James Harper]

In trying to figure out why my Xen Windows PV drivers don't work
properly when the frontend and backend are different bit width modes (eg
32, 64), I figured out that the same is true for Linux domains. I tested
a 32 bit DomU with a 64 bit Dom0, and get a crash almost as soon as the
blkfront driver is started, and a "(XEN) grant_table.c:264:d0 Bad flags
(0) or dom (0). (expected dom 0)" message logged in 'xm dmesg'.

My suspicion then is that the Debian kernel's blkback driver doesn't pay
attention to the 'protocol' set by the frontend, even though blkfront
driver included in the Debian kernel does actually set it.

Can anyone confirm this behaviour?

Thanks

James

Re: Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

[Daniel P. Berrange]

On Sat, Feb 02, 2008 at 09:25:27AM +1100, James Harper wrote:
> In trying to figure out why my Xen Windows PV drivers don't work
> properly when the frontend and backend are different bit width modes (eg
> 32, 64), I figured out that the same is true for Linux domains. I tested
> a 32 bit DomU with a 64 bit Dom0, and get a crash almost as soon as the
> blkfront driver is started, and a "(XEN) grant_table.c:264:d0 Bad flags
> (0) or dom (0). (expected dom 0)" message logged in 'xm dmesg'.
> 
> My suspicion then is that the Debian kernel's blkback driver doesn't pay
> attention to the 'protocol' set by the frontend, even though blkfront
> driver included in the Debian kernel does actually set it.

Depends what Xen version their kernel / userspace is based on. IIRC you
need to have at least Xen 3.1.1  to get 32-on-64 working properly - the
compatability 'protocol' stuff didn't make the original 3.1.0 release

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

RE: Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

[James Harper]

> >
> > My suspicion then is that the Debian kernel's blkback driver doesn't
pay
> > attention to the 'protocol' set by the frontend, even though
blkfront
> > driver included in the Debian kernel does actually set it.
> 
> Depends what Xen version their kernel / userspace is based on. IIRC
you
> need to have at least Xen 3.1.1  to get 32-on-64 working properly -
the
> compatability 'protocol' stuff didn't make the original 3.1.0 release
> 

I am using 3.1.1, but I think the Debian kernel is 3.1.0 vintage.

And I was incorrect when I said that blkfront is setting 'x86_NN-abi'.
That is being done by the xen userspace stuff. It's all making sense
now!

Curiously, the HVM stuff works perfectly with the single exception of
vbd, and only because of the alignment issues in the two related
structs.

I think I'll fudge it and release two versions of the PV drivers, one to
use when Dom0 is 32 bit and one to use when Dom0 is 64 bit.

James

Re: Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

[Ralph Passgang]

On 01.02.08 23:40, "James Harper" <james.harper@bendigoit.com.au> wrote:

>>> 
>>> My suspicion then is that the Debian kernel's blkback driver doesn't
> pay
>>> attention to the 'protocol' set by the frontend, even though
> blkfront
>>> driver included in the Debian kernel does actually set it.
>> 
>> Depends what Xen version their kernel / userspace is based on. IIRC
> you
>> need to have at least Xen 3.1.1  to get 32-on-64 working properly -
> the
>> compatability 'protocol' stuff didn't make the original 3.1.0 release
>> 
> 
> I am using 3.1.1, but I think the Debian kernel is 3.1.0 vintage.

Which Debian Kernel?

The 2.6.18-5-xen-686/amd64 kernel is quite old and based on some redhat
kernel patch from around Xen 3.0.x.

It's definitly too old for mixing 64bit and 32bit machines (pv) up.

Please use a newer kernel for both dom0 and domU, if you need help with
that, let me know.
 
> And I was incorrect when I said that blkfront is setting 'x86_NN-abi'.
> That is being done by the xen userspace stuff. It's all making sense
> now!
> 
> Curiously, the HVM stuff works perfectly with the single exception of
> vbd, and only because of the alignment issues in the two related
> structs.
> 
> I think I'll fudge it and release two versions of the PV drivers, one to
> use when Dom0 is 32 bit and one to use when Dom0 is 64 bit.
> 
> James
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: Debian kernel doesn't support DomU bit width != Dom0 bit width for PV drivers?

[Dan Magenheimer]

Could be totally unrelated, but (despite the name of the patch)
this might be relevant:

http://lists.xensource.com/archives/html/xen-devel/2008-01/msg00509.html

Mixed bit-width-mode dom0 vs domU probably has some lurking bugs.
Oracle has pounded on 64-bit domU on 32-bit dom0 quite a bit,
but not vice-versa.

Note that a 64-bit dom0 isn't required to run 64-bit guests,
just a 64-bit Xen.  As a result, I don't know if any commercial
products run a 64-bit dom0, so the 32-on-64 bugs probably haven't
been shaken out.

Dan

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of 
> James Harper
> Sent: Friday, February 01, 2008 3:25 PM
> To: xen-devel
> Subject: [Xen-devel] Debian kernel doesn't support DomU bit width !=
> Dom0 bit width for PV drivers?
> 
> 
> In trying to figure out why my Xen Windows PV drivers don't work
> properly when the frontend and backend are different bit 
> width modes (eg
> 32, 64), I figured out that the same is true for Linux 
> domains. I tested
> a 32 bit DomU with a 64 bit Dom0, and get a crash almost as 
> soon as the
> blkfront driver is started, and a "(XEN) grant_table.c:264:d0 
> Bad flags
> (0) or dom (0). (expected dom 0)" message logged in 'xm dmesg'.
> 
> My suspicion then is that the Debian kernel's blkback driver 
> doesn't pay
> attention to the 'protocol' set by the frontend, even though blkfront
> driver included in the Debian kernel does actually set it.
> 
> Can anyone confirm this behaviour?
> 
> Thanks
> 
> James
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

RE: Debian kernel doesn't support DomU bit width != Dom0bit width for PV drivers?

[James Harper]

> >
> > I am using 3.1.1, but I think the Debian kernel is 3.1.0 vintage.
> 
> Which Debian Kernel?
> 
> The 2.6.18-5-xen-686/amd64 kernel is quite old and based on some redhat
> kernel patch from around Xen 3.0.x.
> 
> It's definitly too old for mixing 64bit and 32bit machines (pv) up.
> 
> Please use a newer kernel for both dom0 and domU, if you need help with
> that, let me know.

Déjà-vu? I think the list is duplicating things again. I'm sure I have received this email previously...

Anyway, I got it working by putting some logic in the frontend to detect if the backend hasn't detected the frontend's bit width, and we switch on the fly.

James