status of PV drivers in Xen 4.0

status of PV drivers in Xen 4.0

[Guillaume Rousse]

Hello.

The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
still refers to $(objtree)/include/linux/autoconf.h, instead of
$(objtree)/include/generated/autoconf.h in recent kernels.

Is there any plan to update it to a more recent kernel, as xen 4.0 has
switched to a newer pvops tree ?
-- 
BOFH excuse #278:

The Dilithium Crystals need to be rotated.

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
> Hello.
> 
> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
> still refers to $(objtree)/include/linux/autoconf.h, instead of
> $(objtree)/include/generated/autoconf.h in recent kernels.
> 

I believe it should compile with 2.6.27 aswell.. or if it doesn't,
please let us know.

See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
http://wp.colliertech.org/cj/?p=653

If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
then we should definitely add the (possible) patches from Novell.

Does anyone know if the 2.6.3x Novell xenified kernels have PV-on-HVM drivers? 

> Is there any plan to update it to a more recent kernel, as xen 4.0 has
> switched to a newer pvops tree ?
>

There's ongoing work to get the PV-on-HVM drivers accepted to upstream kernel.org
Linux kernel.

-- Pasi

Re: status of PV drivers in Xen 4.0

[Jan Beulich]

>>> Pasi Kärkkäinen<pasik@iki.fi> 30.03.10 08:04 >>>
>Does anyone know if the 2.6.3x Novell xenified kernels have PV-on-HVM drivers? 

.32 certainly does (but still uses autoconf.h). .33/.34 likely weren't built
so far, but we will in any case fix this up once we get there.

Jan

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Tue, Mar 30, 2010 at 08:49:16AM +0100, Jan Beulich wrote:
> >>> Pasi Kärkkäinen<pasik@iki.fi> 30.03.10 08:04 >>>
> >Does anyone know if the 2.6.3x Novell xenified kernels have PV-on-HVM drivers? 
> 
> .32 certainly does (but still uses autoconf.h). .33/.34 likely weren't built
> so far, but we will in any case fix this up once we get there.
>

Great, thanks for the info!

-- Pasi
 

Re: status of PV drivers in Xen 4.0

[Guillaume Rousse]

[-- Attachment #1.1: Type: text/plain, Size: 583 bytes --]

Le 30/03/2010 09:49, Jan Beulich a écrit :
>>>> Pasi Kärkkäinen<pasik@iki.fi> 30.03.10 08:04 >>>
>> Does anyone know if the 2.6.3x Novell xenified kernels have PV-on-HVM drivers? 
> 
> .32 certainly does (but still uses autoconf.h). .33/.34 likely weren't built
> so far, but we will in any case fix this up once we get there.
Just to be sure: does the suse xen kernel directly includes those
additional modules, or is it rather a suitable tree for building them
from xen packages ?
-- 
BOFH excuse #115:

your keyboard's space bar is generating spurious keycodes.


[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Jan Beulich]

>>> Guillaume Rousse <Guillaume.Rousse@inria.fr> 30.03.10 13:27 >>>
>Le 30/03/2010 09:49, Jan Beulich a écrit :
>>>>> Pasi Kärkkäinen<pasik@iki.fi> 30.03.10 08:04 >>>
>>> Does anyone know if the 2.6.3x Novell xenified kernels have PV-on-HVM drivers? 
>> 
>> .32 certainly does (but still uses autoconf.h). .33/.34 likely weren't built
>> so far, but we will in any case fix this up once we get there.
>Just to be sure: does the suse xen kernel directly includes those
>additional modules, or is it rather a suitable tree for building them
>from xen packages ?

Both - they come pre-compiled in KMPs and the tree is also suitable
for building.

Jan

Re: status of PV drivers in Xen 4.0

[Guillaume Rousse]

[-- Attachment #1.1: Type: text/plain, Size: 590 bytes --]

Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
>> Is there any plan to update it to a more recent kernel, as xen 4.0 has
>> switched to a newer pvops tree ?
>>
> 
> There's ongoing work to get the PV-on-HVM drivers accepted to upstream kernel.org
> Linux kernel.
I saw the patches announced there. Also, I saw the announcement in
2.6.33 changelog, but I don't see the exact relationship with the
4 drivers present in xen archive:
http://kernelnewbies.org/Linux_2_6_33#head-be694b031a673ddf43328f04a1cc5c781467bd17
-- 
BOFH excuse #188:

..disk or the processor is on fire.


[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Tue, Mar 30, 2010 at 10:17:01AM +0200, Guillaume Rousse wrote:
> Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
> >> Is there any plan to update it to a more recent kernel, as xen 4.0 has
> >> switched to a newer pvops tree ?
> >>
> > 
> > There's ongoing work to get the PV-on-HVM drivers accepted to upstream kernel.org
> > Linux kernel.
> I saw the patches announced there. Also, I saw the announcement in
> 2.6.33 changelog, but I don't see the exact relationship with the
> 4 drivers present in xen archive:
> http://kernelnewbies.org/Linux_2_6_33#head-be694b031a673ddf43328f04a1cc5c781467bd17

I think that "Support for Xen PV-on-HVM guests" is related to KVM, not to Xen.

-- Pasi

Re: status of PV drivers in Xen 4.0

[Guillaume Rousse]

Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
> On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
>> Hello.
>>
>> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
>> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
>> still refers to $(objtree)/include/linux/autoconf.h, instead of
>> $(objtree)/include/generated/autoconf.h in recent kernels.
>>
> 
> I believe it should compile with 2.6.27 aswell.. or if it doesn't,
> please let us know.
> 
> See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
> http://wp.colliertech.org/cj/?p=653
> 
> If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
> then we should definitely add the (possible) patches from Novell.
Whatever XenLinux tree I tried (xensource 2.6.18, or novell 2.6.32), it
fails to build for 2.6.33. It's probably a trivial error, such as a
missing header file, but I'm really not enough kernel-aware to find
which one:

[guillomovitch@n2 linux-2.6]$ make -C
/lib/modules/2.6.33.1-desktop-2mnb/build M=$PWD modules
make: entrant dans le répertoire « /usr/src/linux-2.6.33.1-desktop-2mnb »
  CC [M]
/home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o
In file included from
/home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:52:
include/xen/interface/memory.h:32: error: expected
specifier-qualifier-list before 'GUEST_HANDLE'
include/xen/interface/memory.h:53: warning: data definition has no type
or storage class
include/xen/interface/memory.h:53: warning: type defaults to 'int' in
declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
include/xen/interface/memory.h:53: warning: parameter names (without
types) in function declaration

BTW, the pvops-based 2.6.32.11 kernel isn't a suitable xenlinux tree, it
misses needed files.
-- 
BOFH excuse #284:

Electrons on a bender

Re: status of PV drivers in Xen 4.0

[Andrew Lyon]

On Mon, Apr 5, 2010 at 5:50 PM, Guillaume Rousse
<Guillaume.Rousse@inria.fr> wrote:
> Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
>> On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
>>> Hello.
>>>
>>> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
>>> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
>>> still refers to $(objtree)/include/linux/autoconf.h, instead of
>>> $(objtree)/include/generated/autoconf.h in recent kernels.
>>>
>>
>> I believe it should compile with 2.6.27 aswell.. or if it doesn't,
>> please let us know.
>>
>> See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
>> http://wp.colliertech.org/cj/?p=653
>>
>> If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
>> then we should definitely add the (possible) patches from Novell.
> Whatever XenLinux tree I tried (xensource 2.6.18, or novell 2.6.32), it
> fails to build for 2.6.33. It's probably a trivial error, such as a
> missing header file, but I'm really not enough kernel-aware to find
> which one:
>

Have you been successful building the drivers for .31 or .32?

I'd like to make a set of patches available to apply to Xen 3.4.2 or
4.0 to enable building the drivers with recent kernels and the forward
ported kernels I usually use, but while I've been able to rebuild them
on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
work out which patches need to be applied and what else the rpm build
process does to enable compilation.

Any pointers would be much appreciated.

Andy

> [guillomovitch@n2 linux-2.6]$ make -C
> /lib/modules/2.6.33.1-desktop-2mnb/build M=$PWD modules
> make: entrant dans le répertoire « /usr/src/linux-2.6.33.1-desktop-2mnb »
>  CC [M]
> /home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o
> In file included from
> /home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:52:
> include/xen/interface/memory.h:32: error: expected
> specifier-qualifier-list before 'GUEST_HANDLE'
> include/xen/interface/memory.h:53: warning: data definition has no type
> or storage class
> include/xen/interface/memory.h:53: warning: type defaults to 'int' in
> declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
> include/xen/interface/memory.h:53: warning: parameter names (without
> types) in function declaration
>
> BTW, the pvops-based 2.6.32.11 kernel isn't a suitable xenlinux tree, it
> misses needed files.
> --
> BOFH excuse #284:
>
> Electrons on a bender
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Sat, Apr 10, 2010 at 01:31:08PM +0100, Andrew Lyon wrote:
> On Mon, Apr 5, 2010 at 5:50 PM, Guillaume Rousse
> <Guillaume.Rousse@inria.fr> wrote:
> > Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
> >> On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
> >>> Hello.
> >>>
> >>> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
> >>> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
> >>> still refers to $(objtree)/include/linux/autoconf.h, instead of
> >>> $(objtree)/include/generated/autoconf.h in recent kernels.
> >>>
> >>
> >> I believe it should compile with 2.6.27 aswell.. or if it doesn't,
> >> please let us know.
> >>
> >> See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
> >> http://wp.colliertech.org/cj/?p=653
> >>
> >> If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
> >> then we should definitely add the (possible) patches from Novell.
> > Whatever XenLinux tree I tried (xensource 2.6.18, or novell 2.6.32), it
> > fails to build for 2.6.33. It's probably a trivial error, such as a
> > missing header file, but I'm really not enough kernel-aware to find
> > which one:
> >
> 
> Have you been successful building the drivers for .31 or .32?
> 
> I'd like to make a set of patches available to apply to Xen 3.4.2 or
> 4.0 to enable building the drivers with recent kernels and the forward
> ported kernels I usually use, but while I've been able to rebuild them
> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
> work out which patches need to be applied and what else the rpm build
> process does to enable compilation.
> 
> Any pointers would be much appreciated.
> 

Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
should have pv-on-hvm drivers included? 

-- Pasi

> Andy
> 
> > [guillomovitch@n2 linux-2.6]$ make -C
> > /lib/modules/2.6.33.1-desktop-2mnb/build M=$PWD modules
> > make: entrant dans le répertoire « /usr/src/linux-2.6.33.1-desktop-2mnb »
> >  CC [M]
> > /home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o
> > In file included from
> > /home/guillomovitch/cooker/xen/BUILD/xen-4.0-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:52:
> > include/xen/interface/memory.h:32: error: expected
> > specifier-qualifier-list before 'GUEST_HANDLE'
> > include/xen/interface/memory.h:53: warning: data definition has no type
> > or storage class
> > include/xen/interface/memory.h:53: warning: type defaults to 'int' in
> > declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
> > include/xen/interface/memory.h:53: warning: parameter names (without
> > types) in function declaration
> >
> > BTW, the pvops-based 2.6.32.11 kernel isn't a suitable xenlinux tree, it
> > misses needed files.
> > --
> > BOFH excuse #284:
> >
> > Electrons on a bender
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Andrew Lyon]

On Sat, Apr 10, 2010 at 1:58 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Sat, Apr 10, 2010 at 01:31:08PM +0100, Andrew Lyon wrote:
>> On Mon, Apr 5, 2010 at 5:50 PM, Guillaume Rousse
>> <Guillaume.Rousse@inria.fr> wrote:
>> > Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
>> >> On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
>> >>> Hello.
>> >>>
>> >>> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
>> >>> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
>> >>> still refers to $(objtree)/include/linux/autoconf.h, instead of
>> >>> $(objtree)/include/generated/autoconf.h in recent kernels.
>> >>>
>> >>
>> >> I believe it should compile with 2.6.27 aswell.. or if it doesn't,
>> >> please let us know.
>> >>
>> >> See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
>> >> http://wp.colliertech.org/cj/?p=653
>> >>
>> >> If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
>> >> then we should definitely add the (possible) patches from Novell.
>> > Whatever XenLinux tree I tried (xensource 2.6.18, or novell 2.6.32), it
>> > fails to build for 2.6.33. It's probably a trivial error, such as a
>> > missing header file, but I'm really not enough kernel-aware to find
>> > which one:
>> >
>>
>> Have you been successful building the drivers for .31 or .32?
>>
>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
>> 4.0 to enable building the drivers with recent kernels and the forward
>> ported kernels I usually use, but while I've been able to rebuild them
>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
>> work out which patches need to be applied and what else the rpm build
>> process does to enable compilation.
>>
>> Any pointers would be much appreciated.
>>
>
> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
> should have pv-on-hvm drivers included?
>
> -- Pasi
>
>> Andy

Yes, thats where I take the 2.6.32 kernel patches from, .31 from
openSUSE-11.2 and .32 from SLE11-SP1, but they both fail to compile in
similar ways, I followed a similar process to
http://wp.colliertech.org/cj/?p=653 :

 pwd
/home/admin/hg/xen-3.4-testing.hg
cd unmodified_drivers/linux-2.6/
ubermicro linux-2.6 # XEN=/home/admin/hg/xen-3.4-testing.hg/
XL=/usr/src/linux-2.6.32-xen/ ./mkbuildtree x86_64
Xen tree: /home/admin/hg/xen-3.4-testing.hg
Linux tree: /usr/src/linux-2.6.32-xen
/usr/src/linux-2.6.32-xen/include/xen/interface:
interface: File exists

To start with I tried to build the modules for the same kernel:

make -C /usr/src/linux-2.6.32-xen/ M=$PWD modules
make: Entering directory `/usr/src/linux-2.6.32-xen'
  CC [M]  /home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o
<command-line>: warning: "__XEN_INTERFACE_VERSION__" redefined
<command-line>: warning: this is the location of the previous definition
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:
In function 'balloon_init':
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
error: implicit declaration of function 'pagevec_init'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
error: 'free_pagevec' undeclared (first use in this function)
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
error: (Each undeclared identifier is reported only once
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
error: for each function it appears in.)
make[2]: *** [/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o]
Error 1
make[1]: *** [/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon]
Error 2
make: *** [_module_/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6]
Error 2
make: Leaving directory `/usr/src/linux-2.6.32-xen'


But it also fails with a non-xen kernel:

 make -C /usr/src/linux-2.6.32-gentoo-r8/ M=$PWD modules
make: Entering directory `/usr/src/linux-2.6.32-gentoo-r8'

  WARNING: Symbol version dump /usr/src/linux-2.6.32-gentoo-r8/Module.symvers
           is missing; modules will have no dependencies and modversions.

  CC [M]  /home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o
In file included from
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:51:
include/xen/interface/memory.h:32: error: expected
specifier-qualifier-list before 'GUEST_HANDLE'
include/xen/interface/memory.h:53: warning: data definition has no
type or storage class
include/xen/interface/memory.h:53: warning: type defaults to 'int' in
declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
include/xen/interface/memory.h:53: warning: parameter names (without
types) in function declaration
include/xen/interface/memory.h:89: error: expected
specifier-qualifier-list before 'GUEST_HANDLE'
include/xen/interface/memory.h:97: warning: data definition has no
type or storage class
include/xen/interface/memory.h:97: warning: type defaults to 'int' in
declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
include/xen/interface/memory.h:97: warning: parameter names (without
types) in function declaration
include/xen/interface/memory.h:107: error: expected
specifier-qualifier-list before 'domid_t'
include/xen/interface/memory.h:120: warning: data definition has no
type or storage class
include/xen/interface/memory.h:120: warning: type defaults to 'int' in
declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
include/xen/interface/memory.h:120: warning: parameter names (without
types) in function declaration
include/xen/interface/memory.h:129: error: expected
specifier-qualifier-list before 'domid_t'
include/xen/interface/memory.h:143: warning: data definition has no
type or storage class
include/xen/interface/memory.h:143: warning: type defaults to 'int' in
declaration of 'DEFINE_GUEST_HANDLE_STRUCT'
include/xen/interface/memory.h:143: warning: parameter names (without
types) in function declaration
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:
In function 'increase_reservation':
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:286:
error: unknown field 'address_bits' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:286:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:286:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:287:
error: unknown field 'extent_order' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:287:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:287:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:288:
error: unknown field 'domid' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:289:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:289:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:303:
error: 'struct xen_memory_reservation' has no member named
'extent_start'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:304:
error: 'struct xen_memory_reservation' has no member named
'nr_extents'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:305:
error: implicit declaration of function 'HYPERVISOR_memory_op'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:
In function 'decrease_reservation':
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:364:
error: unknown field 'address_bits' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:364:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:364:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:365:
error: unknown field 'extent_order' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:365:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:365:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:366:
error: unknown field 'domid' specified in initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:367:
warning: excess elements in struct initializer
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:367:
warning: (near initialization for 'reservation')
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:384:
error: implicit declaration of function 'scrub_pages'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:414:
error: 'struct xen_memory_reservation' has no member named
'extent_start'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:415:
error: 'struct xen_memory_reservation' has no member named
'nr_extents'
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:
In function 'balloon_init':
/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.c:577:
error: implicit declaration of function 'is_running_on_xen'
make[2]: *** [/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon/balloon.o]
Error 1
make[1]: *** [/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6/balloon]
Error 2
make: *** [_module_/home/admin/hg/xen-3.4-testing.hg/unmodified_drivers/linux-2.6]
Error 2
make: Leaving directory `/usr/src/linux-2.6.32-gentoo-r8'
ubermicro linux-2.6 #


Andy

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Sat, Apr 10, 2010 at 02:17:57PM +0100, Andrew Lyon wrote:
> On Sat, Apr 10, 2010 at 1:58 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > On Sat, Apr 10, 2010 at 01:31:08PM +0100, Andrew Lyon wrote:
> >> On Mon, Apr 5, 2010 at 5:50 PM, Guillaume Rousse
> >> <Guillaume.Rousse@inria.fr> wrote:
> >> > Le 30/03/2010 08:04, Pasi Kärkkäinen a écrit :
> >> >> On Mon, Mar 29, 2010 at 11:02:09PM +0200, Guillaume Rousse wrote:
> >> >>> Hello.
> >> >>>
> >> >>> The directory 'unmodified_drivers', in Xen-4.0rc8, seems to still
> >> >>> requires a 2.6.18 XenLinux kernel to build. For instance, overrides.mk
> >> >>> still refers to $(objtree)/include/linux/autoconf.h, instead of
> >> >>> $(objtree)/include/generated/autoconf.h in recent kernels.
> >> >>>
> >> >>
> >> >> I believe it should compile with 2.6.27 aswell.. or if it doesn't,
> >> >> please let us know.
> >> >>
> >> >> See here for PV-on-HVM drivers building instructions for SLES11 2.6.27:
> >> >> http://wp.colliertech.org/cj/?p=653
> >> >>
> >> >> If the unmodified_drivers from Xen 4.0 doesn't build for 2.6.27,
> >> >> then we should definitely add the (possible) patches from Novell.
> >> > Whatever XenLinux tree I tried (xensource 2.6.18, or novell 2.6.32), it
> >> > fails to build for 2.6.33. It's probably a trivial error, such as a
> >> > missing header file, but I'm really not enough kernel-aware to find
> >> > which one:
> >> >
> >>
> >> Have you been successful building the drivers for .31 or .32?
> >>
> >> I'd like to make a set of patches available to apply to Xen 3.4.2 or
> >> 4.0 to enable building the drivers with recent kernels and the forward
> >> ported kernels I usually use, but while I've been able to rebuild them
> >> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
> >> work out which patches need to be applied and what else the rpm build
> >> process does to enable compilation.
> >>
> >> Any pointers would be much appreciated.
> >>
> >
> > Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
> > should have pv-on-hvm drivers included?
> >
> > -- Pasi
> >
> >> Andy
> 
> Yes, thats where I take the 2.6.32 kernel patches from, .31 from
> openSUSE-11.2 and .32 from SLE11-SP1, but they both fail to compile in
> similar ways, I followed a similar process to
> http://wp.colliertech.org/cj/?p=653 :
> 
>  pwd
> /home/admin/hg/xen-3.4-testing.hg
> cd unmodified_drivers/linux-2.6/
> ubermicro linux-2.6 # XEN=/home/admin/hg/xen-3.4-testing.hg/
> XL=/usr/src/linux-2.6.32-xen/ ./mkbuildtree x86_64
> Xen tree: /home/admin/hg/xen-3.4-testing.hg
> Linux tree: /usr/src/linux-2.6.32-xen
> /usr/src/linux-2.6.32-xen/include/xen/interface:
> interface: File exists
> 
> To start with I tried to build the modules for the same kernel:
> 

Hmm..

see these emails:

http://lists.xensource.com/archives/html/xen-devel/2010-03/msg01597.html
and
http://lists.xensource.com/archives/html/xen-devel/2010-03/msg01570.html

So, somehow they should build :) 

Maybe we should take a look at the sles/opensuse rpms? 

-- Pasi

Re: status of PV drivers in Xen 4.0

[Guillaume Rousse]

[-- Attachment #1.1: Type: text/plain, Size: 1425 bytes --]

Le 10/04/2010 14:58, Pasi Kärkkäinen a écrit :
>> Have you been successful building the drivers for .31 or .32?
I just tried 2.6.33 so far, without success.

>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
>> 4.0 to enable building the drivers with recent kernels and the forward
>> ported kernels I usually use, but while I've been able to rebuild them
>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
>> work out which patches need to be applied and what else the rpm build
>> process does to enable compilation.
>>
>> Any pointers would be much appreciated.
>>
> 
> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
> should have pv-on-hvm drivers included?
Yes, that's Jan Beulich said on this list. However, I've buid a
2.6.32.11 kernel, with all xen patches available from SLE11-SP1 (as well
as a few others needed for applying xen patches properly), and used the
corresponding defconfig file. But I can't find any reference to those
drivers, at least with the names given in README file of xen tarball
(xen-platform-pci.ko, xen-vbd.ko, xen-vnif.ko and xen-balloon.ko).
That's probably just because of a name change.

Anyway (unless I'm wrong), the actual usefulness of pv-on-hvm is on
standard linux kernels, not on dom0 itself, and the issue seems to be on
xen Makefile side.
-- 
BOFH excuse #298:

Not enough interrupts


[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Andrew Lyon]

On Mon, Apr 12, 2010 at 12:27 PM, Guillaume Rousse
<Guillaume.Rousse@inria.fr> wrote:
> Le 10/04/2010 14:58, Pasi Kärkkäinen a écrit :
>>> Have you been successful building the drivers for .31 or .32?
> I just tried 2.6.33 so far, without success.
>
>>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
>>> 4.0 to enable building the drivers with recent kernels and the forward
>>> ported kernels I usually use, but while I've been able to rebuild them
>>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
>>> work out which patches need to be applied and what else the rpm build
>>> process does to enable compilation.
>>>
>>> Any pointers would be much appreciated.
>>>
>>
>> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
>> should have pv-on-hvm drivers included?
> Yes, that's Jan Beulich said on this list. However, I've buid a
> 2.6.32.11 kernel, with all xen patches available from SLE11-SP1 (as well
> as a few others needed for applying xen patches properly), and used the
> corresponding defconfig file. But I can't find any reference to those
> drivers, at least with the names given in README file of xen tarball
> (xen-platform-pci.ko, xen-vbd.ko, xen-vnif.ko and xen-balloon.ko).
> That's probably just because of a name change.
>
> Anyway (unless I'm wrong), the actual usefulness of pv-on-hvm is on
> standard linux kernels, not on dom0 itself, and the issue seems to be on
> xen Makefile side.
> --

I've been looking at this over the weekend and I've made some
progress, you are correct about the usefulness of the pv-on-hvm
drivers, however you need to bear in mind that the openSUSE/SLES11
kernels *always* include the xen patches even if they are not
activated for a particular kernel flavour (e.g. desktop, debug etc),
so there are some differences in how the drivers are built,
specifically the build process uses header files from the xenified
kernel instead of from Xen itself.

You will not find the pv-on-hvm drivers in the kernel configuration
options of any xenified kernel, they are built separately as part of
Xen, I decided to have a look at 2.6.31 so I downloaded the latest Xen
source rpm for openSUSE 11.2
(http://download.opensuse.org/source/distribution/11.2/repo/oss/suse/src/xen-3.4.1_19718_04-2.1.src.rpm),
you need to extract the contents of the rpm, then extract the
xen-3.4.1-testing-src.tar.bz2, then you need to apply the same patches
that the rpm build process would apply, you can work out which patches
by reading xen.spec or by running rpmbuild -bb xen.spec on a opensuse
system and logging the console output during the build process, then
parsing the log for patch commands and running them on the extracted
xen source tree.

The unmodified_drivers build process is designed to work with a
separate kernel "objects" folder, so instead of configuring and
compiling your kernel in-place you need to do something like this,
note that while this step uses a xenified kernel you should not enable
Xen in the processor type options, configure the kernel as you would
for the hvm guest.

cd /usr/src/linux-2.6.31-xen-r14
mkdir ../linux-2.6.31-xen-r14-obj
mv .config ../linux-2.6.31-xen-r14-obj/ (or copy in your usual default
config file)
make mrproper
cd ../linux-2.6.31-xen-r14-obj/
make -C /usr/src/linux-2.6.31-xen-r14 menuconfig
O=/usr/src/linux-2.6.31-xen-r14-obj/
make -C /usr/src/linux-2.6.31-xen-r14 bzImage (or vmlinuz) modules
modules_install O=/usr/src/linux-2.6.31-xen-r14-obj/

Now change to wherever you put the unmodified_drivers/linux-2.6/:

export XL=/usr/src/linux-2.6.31-xen-r14
export XEN=/usr/src/linux-2.6.31-xen-r14/include/xen/

Notice how XEN is set to linux/include/xen ? the Xen documentation
states that it should be set to a folder containing the Xen sources,
I've tried that and it does not work, I think the folder is only used
for header files so they could in theory come from Xen or from a
Xenified kernel, but on openSUSE the kernel include files are used
instead of the Xen ones, and that is the only way that seems to work.

./mkbuildtree
Defaulting to this machine's architecture, x86_64, for linking.
This may be overridden on the command line (i386,x86_64,ia64).
Xen tree: /usr/src/linux-2.6.31-xen-r14/include/xen
Linux tree: /usr/src/linux-2.6.31-xen-r14

You can now compile the drivers:

make -C /usr/src/linux-2.6.31-xen-r14-obj/ modules M=$PWD

If you attempt to do the same process without putting the kernel
object files into a separate folder the build will fail, I believe
this is because declarations in the kernel override those in the
unmodified_drivers.

Likewise the process fails if you try to use a generic kernel source
tree which has not been patched for Xen dom0, in which case the header
files are missing various Xen declarations required for successful
compilation.

And here is a final proof, Xenified 2.6.31 with all XEN options
disabled, with pv-on-hvm drivers:

localhost ~ # uname -a
Linux localhost 2.6.31-xen-r14 #2 SMP Mon Apr 12 13:34:27 GMT 2010
x86_64 Intel(R) Xeon(R) CPU E5420 @ 2.50GHz GenuineIntel GNU/Linux
localhost ~ # zgrep XEN /proc/config.gz
# CONFIG_X86_64_XEN is not set
localhost ~ # lsmod
Module                  Size  Used by
xen_vnif               28288  0
xen_balloon            14496  1 xen_vnif
xen_vbd                19084  3
xen_platform_pci       78808  3 xen_vnif,xen_balloon,xen_vbd,[permanent]

I'm not sure where to go next with this but I hope this info is useful.

Andy

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Mon, Apr 12, 2010 at 04:40:49PM +0100, Andrew Lyon wrote:
> On Mon, Apr 12, 2010 at 12:27 PM, Guillaume Rousse
> <Guillaume.Rousse@inria.fr> wrote:
> > Le 10/04/2010 14:58, Pasi Kärkkäinen a écrit :
> >>> Have you been successful building the drivers for .31 or .32?
> > I just tried 2.6.33 so far, without success.
> >
> >>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
> >>> 4.0 to enable building the drivers with recent kernels and the forward
> >>> ported kernels I usually use, but while I've been able to rebuild them
> >>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
> >>> work out which patches need to be applied and what else the rpm build
> >>> process does to enable compilation.
> >>>
> >>> Any pointers would be much appreciated.
> >>>
> >>
> >> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
> >> should have pv-on-hvm drivers included?
> > Yes, that's Jan Beulich said on this list. However, I've buid a
> > 2.6.32.11 kernel, with all xen patches available from SLE11-SP1 (as well
> > as a few others needed for applying xen patches properly), and used the
> > corresponding defconfig file. But I can't find any reference to those
> > drivers, at least with the names given in README file of xen tarball
> > (xen-platform-pci.ko, xen-vbd.ko, xen-vnif.ko and xen-balloon.ko).
> > That's probably just because of a name change.
> >
> > Anyway (unless I'm wrong), the actual usefulness of pv-on-hvm is on
> > standard linux kernels, not on dom0 itself, and the issue seems to be on
> > xen Makefile side.
> > --
> 
> I've been looking at this over the weekend and I've made some
> progress, you are correct about the usefulness of the pv-on-hvm
> drivers, however you need to bear in mind that the openSUSE/SLES11
> kernels *always* include the xen patches even if they are not
> activated for a particular kernel flavour (e.g. desktop, debug etc),
> so there are some differences in how the drivers are built,
> specifically the build process uses header files from the xenified
> kernel instead of from Xen itself.
> 
> You will not find the pv-on-hvm drivers in the kernel configuration
> options of any xenified kernel, they are built separately as part of
> Xen, I decided to have a look at 2.6.31 so I downloaded the latest Xen
> source rpm for openSUSE 11.2
> (http://download.opensuse.org/source/distribution/11.2/repo/oss/suse/src/xen-3.4.1_19718_04-2.1.src.rpm),
> you need to extract the contents of the rpm, then extract the
> xen-3.4.1-testing-src.tar.bz2, then you need to apply the same patches
> that the rpm build process would apply, you can work out which patches
> by reading xen.spec or by running rpmbuild -bb xen.spec on a opensuse
> system and logging the console output during the build process, then
> parsing the log for patch commands and running them on the extracted
> xen source tree.
> 

There also the "-bp" option for rpmbuild:

-bp    Executes  the  "%prep"  stage  from  the  spec  file. 
       Normally this involves unpacking the sources and applying any patches.



> The unmodified_drivers build process is designed to work with a
> separate kernel "objects" folder, so instead of configuring and
> compiling your kernel in-place you need to do something like this,
> note that while this step uses a xenified kernel you should not enable
> Xen in the processor type options, configure the kernel as you would
> for the hvm guest.
> 
> cd /usr/src/linux-2.6.31-xen-r14
> mkdir ../linux-2.6.31-xen-r14-obj
> mv .config ../linux-2.6.31-xen-r14-obj/ (or copy in your usual default
> config file)
> make mrproper
> cd ../linux-2.6.31-xen-r14-obj/
> make -C /usr/src/linux-2.6.31-xen-r14 menuconfig
> O=/usr/src/linux-2.6.31-xen-r14-obj/
> make -C /usr/src/linux-2.6.31-xen-r14 bzImage (or vmlinuz) modules
> modules_install O=/usr/src/linux-2.6.31-xen-r14-obj/
> 
> Now change to wherever you put the unmodified_drivers/linux-2.6/:
> 
> export XL=/usr/src/linux-2.6.31-xen-r14
> export XEN=/usr/src/linux-2.6.31-xen-r14/include/xen/
> 
> Notice how XEN is set to linux/include/xen ? the Xen documentation
> states that it should be set to a folder containing the Xen sources,
> I've tried that and it does not work, I think the folder is only used
> for header files so they could in theory come from Xen or from a
> Xenified kernel, but on openSUSE the kernel include files are used
> instead of the Xen ones, and that is the only way that seems to work.
> 
> ./mkbuildtree
> Defaulting to this machine's architecture, x86_64, for linking.
> This may be overridden on the command line (i386,x86_64,ia64).
> Xen tree: /usr/src/linux-2.6.31-xen-r14/include/xen
> Linux tree: /usr/src/linux-2.6.31-xen-r14
> 
> You can now compile the drivers:
> 
> make -C /usr/src/linux-2.6.31-xen-r14-obj/ modules M=$PWD
> 
> If you attempt to do the same process without putting the kernel
> object files into a separate folder the build will fail, I believe
> this is because declarations in the kernel override those in the
> unmodified_drivers.
> 
> Likewise the process fails if you try to use a generic kernel source
> tree which has not been patched for Xen dom0, in which case the header
> files are missing various Xen declarations required for successful
> compilation.
> 
> And here is a final proof, Xenified 2.6.31 with all XEN options
> disabled, with pv-on-hvm drivers:
> 
> localhost ~ # uname -a
> Linux localhost 2.6.31-xen-r14 #2 SMP Mon Apr 12 13:34:27 GMT 2010
> x86_64 Intel(R) Xeon(R) CPU E5420 @ 2.50GHz GenuineIntel GNU/Linux
> localhost ~ # zgrep XEN /proc/config.gz
> # CONFIG_X86_64_XEN is not set
> localhost ~ # lsmod
> Module                  Size  Used by
> xen_vnif               28288  0
> xen_balloon            14496  1 xen_vnif
> xen_vbd                19084  3
> xen_platform_pci       78808  3 xen_vnif,xen_balloon,xen_vbd,[permanent]
> 
> I'm not sure where to go next with this but I hope this info is useful.
> 

Congratulations!

This is valuable information, thanks for going through that.

Can you reproduce this with Xen 4.0.0 ? ie. does opensuse xen have 
some patches related to pv-on-hvm drivers? 

If there are some patches, those should be upstreamed to Xen.

Also we could try to make the compilation process easier.. somehow.

-- Pasi

Re: status of PV drivers in Xen 4.0

[Andrew Lyon]

On Mon, Apr 12, 2010 at 4:50 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Mon, Apr 12, 2010 at 04:40:49PM +0100, Andrew Lyon wrote:
>> On Mon, Apr 12, 2010 at 12:27 PM, Guillaume Rousse
>> <Guillaume.Rousse@inria.fr> wrote:
>> > Le 10/04/2010 14:58, Pasi Kärkkäinen a écrit :
>> >>> Have you been successful building the drivers for .31 or .32?
>> > I just tried 2.6.33 so far, without success.
>> >
>> >>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
>> >>> 4.0 to enable building the drivers with recent kernels and the forward
>> >>> ported kernels I usually use, but while I've been able to rebuild them
>> >>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
>> >>> work out which patches need to be applied and what else the rpm build
>> >>> process does to enable compilation.
>> >>>
>> >>> Any pointers would be much appreciated.
>> >>>
>> >>
>> >> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
>> >> should have pv-on-hvm drivers included?
>> > Yes, that's Jan Beulich said on this list. However, I've buid a
>> > 2.6.32.11 kernel, with all xen patches available from SLE11-SP1 (as well
>> > as a few others needed for applying xen patches properly), and used the
>> > corresponding defconfig file. But I can't find any reference to those
>> > drivers, at least with the names given in README file of xen tarball
>> > (xen-platform-pci.ko, xen-vbd.ko, xen-vnif.ko and xen-balloon.ko).
>> > That's probably just because of a name change.
>> >
>> > Anyway (unless I'm wrong), the actual usefulness of pv-on-hvm is on
>> > standard linux kernels, not on dom0 itself, and the issue seems to be on
>> > xen Makefile side.
>> > --
>>
>> I've been looking at this over the weekend and I've made some
>> progress, you are correct about the usefulness of the pv-on-hvm
>> drivers, however you need to bear in mind that the openSUSE/SLES11
>> kernels *always* include the xen patches even if they are not
>> activated for a particular kernel flavour (e.g. desktop, debug etc),
>> so there are some differences in how the drivers are built,
>> specifically the build process uses header files from the xenified
>> kernel instead of from Xen itself.
>>
>> You will not find the pv-on-hvm drivers in the kernel configuration
>> options of any xenified kernel, they are built separately as part of
>> Xen, I decided to have a look at 2.6.31 so I downloaded the latest Xen
>> source rpm for openSUSE 11.2
>> (http://download.opensuse.org/source/distribution/11.2/repo/oss/suse/src/xen-3.4.1_19718_04-2.1.src.rpm),
>> you need to extract the contents of the rpm, then extract the
>> xen-3.4.1-testing-src.tar.bz2, then you need to apply the same patches
>> that the rpm build process would apply, you can work out which patches
>> by reading xen.spec or by running rpmbuild -bb xen.spec on a opensuse
>> system and logging the console output during the build process, then
>> parsing the log for patch commands and running them on the extracted
>> xen source tree.
>>
>
> There also the "-bp" option for rpmbuild:
>
> -bp    Executes  the  "%prep"  stage  from  the  spec  file.
>       Normally this involves unpacking the sources and applying any patches.
>

Noted, that will save time in future :).

>
>
>> The unmodified_drivers build process is designed to work with a
>> separate kernel "objects" folder, so instead of configuring and
>> compiling your kernel in-place you need to do something like this,
>> note that while this step uses a xenified kernel you should not enable
>> Xen in the processor type options, configure the kernel as you would
>> for the hvm guest.
>>
>> cd /usr/src/linux-2.6.31-xen-r14
>> mkdir ../linux-2.6.31-xen-r14-obj
>> mv .config ../linux-2.6.31-xen-r14-obj/ (or copy in your usual default
>> config file)
>> make mrproper
>> cd ../linux-2.6.31-xen-r14-obj/
>> make -C /usr/src/linux-2.6.31-xen-r14 menuconfig
>> O=/usr/src/linux-2.6.31-xen-r14-obj/
>> make -C /usr/src/linux-2.6.31-xen-r14 bzImage (or vmlinuz) modules
>> modules_install O=/usr/src/linux-2.6.31-xen-r14-obj/
>>
>> Now change to wherever you put the unmodified_drivers/linux-2.6/:
>>
>> export XL=/usr/src/linux-2.6.31-xen-r14
>> export XEN=/usr/src/linux-2.6.31-xen-r14/include/xen/
>>
>> Notice how XEN is set to linux/include/xen ? the Xen documentation
>> states that it should be set to a folder containing the Xen sources,
>> I've tried that and it does not work, I think the folder is only used
>> for header files so they could in theory come from Xen or from a
>> Xenified kernel, but on openSUSE the kernel include files are used
>> instead of the Xen ones, and that is the only way that seems to work.
>>
>> ./mkbuildtree
>> Defaulting to this machine's architecture, x86_64, for linking.
>> This may be overridden on the command line (i386,x86_64,ia64).
>> Xen tree: /usr/src/linux-2.6.31-xen-r14/include/xen
>> Linux tree: /usr/src/linux-2.6.31-xen-r14
>>
>> You can now compile the drivers:
>>
>> make -C /usr/src/linux-2.6.31-xen-r14-obj/ modules M=$PWD
>>
>> If you attempt to do the same process without putting the kernel
>> object files into a separate folder the build will fail, I believe
>> this is because declarations in the kernel override those in the
>> unmodified_drivers.
>>
>> Likewise the process fails if you try to use a generic kernel source
>> tree which has not been patched for Xen dom0, in which case the header
>> files are missing various Xen declarations required for successful
>> compilation.
>>
>> And here is a final proof, Xenified 2.6.31 with all XEN options
>> disabled, with pv-on-hvm drivers:
>>
>> localhost ~ # uname -a
>> Linux localhost 2.6.31-xen-r14 #2 SMP Mon Apr 12 13:34:27 GMT 2010
>> x86_64 Intel(R) Xeon(R) CPU E5420 @ 2.50GHz GenuineIntel GNU/Linux
>> localhost ~ # zgrep XEN /proc/config.gz
>> # CONFIG_X86_64_XEN is not set
>> localhost ~ # lsmod
>> Module                  Size  Used by
>> xen_vnif               28288  0
>> xen_balloon            14496  1 xen_vnif
>> xen_vbd                19084  3
>> xen_platform_pci       78808  3 xen_vnif,xen_balloon,xen_vbd,[permanent]
>>
>> I'm not sure where to go next with this but I hope this info is useful.
>>
>
> Congratulations!
>
> This is valuable information, thanks for going through that.
>
> Can you reproduce this with Xen 4.0.0 ? ie. does opensuse xen have
> some patches related to pv-on-hvm drivers?

I believe SLES11SP1 will have Xen 4.0 but I have been unable to find
anywhere to download srpms, apparently they are on dvd2 but as it is
still in beta there are no iso images either, however opensuse factory
does have a Xen 4.0 srpm:

http://download.opensuse.org/factory/repo/src-oss/suse/src/xen-4.0.0_20978_01-1.4.src.rpm

So the same process could be followed using that, I'm going to give it
a try tomorrow.

>
> If there are some patches, those should be upstreamed to Xen.

Hmm, the patches may simply be there to allow building using the
/usr/src/linux/include/xen headers instead of /usr/src/xen/ , and as
xensource does not have a newer "classic" kernel than 2.6.18 why would
they upsteam them.

>
> Also we could try to make the compilation process easier.. somehow.

Indeed :).

>
> -- Pasi
>
>

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Mon, Apr 12, 2010 at 05:05:37PM +0100, Andrew Lyon wrote:
> >
> >> The unmodified_drivers build process is designed to work with a
> >> separate kernel "objects" folder, so instead of configuring and
> >> compiling your kernel in-place you need to do something like this,
> >> note that while this step uses a xenified kernel you should not enable
> >> Xen in the processor type options, configure the kernel as you would
> >> for the hvm guest.
> >>
> >> cd /usr/src/linux-2.6.31-xen-r14
> >> mkdir ../linux-2.6.31-xen-r14-obj
> >> mv .config ../linux-2.6.31-xen-r14-obj/ (or copy in your usual default
> >> config file)
> >> make mrproper
> >> cd ../linux-2.6.31-xen-r14-obj/
> >> make -C /usr/src/linux-2.6.31-xen-r14 menuconfig
> >> O=/usr/src/linux-2.6.31-xen-r14-obj/
> >> make -C /usr/src/linux-2.6.31-xen-r14 bzImage (or vmlinuz) modules
> >> modules_install O=/usr/src/linux-2.6.31-xen-r14-obj/
> >>
> >> Now change to wherever you put the unmodified_drivers/linux-2.6/:
> >>
> >> export XL=/usr/src/linux-2.6.31-xen-r14
> >> export XEN=/usr/src/linux-2.6.31-xen-r14/include/xen/
> >>
> >> Notice how XEN is set to linux/include/xen ? the Xen documentation
> >> states that it should be set to a folder containing the Xen sources,
> >> I've tried that and it does not work, I think the folder is only used
> >> for header files so they could in theory come from Xen or from a
> >> Xenified kernel, but on openSUSE the kernel include files are used
> >> instead of the Xen ones, and that is the only way that seems to work.
> >>
> >> ./mkbuildtree
> >> Defaulting to this machine's architecture, x86_64, for linking.
> >> This may be overridden on the command line (i386,x86_64,ia64).
> >> Xen tree: /usr/src/linux-2.6.31-xen-r14/include/xen
> >> Linux tree: /usr/src/linux-2.6.31-xen-r14
> >>
> >> You can now compile the drivers:
> >>
> >> make -C /usr/src/linux-2.6.31-xen-r14-obj/ modules M=$PWD
> >>
> >> If you attempt to do the same process without putting the kernel
> >> object files into a separate folder the build will fail, I believe
> >> this is because declarations in the kernel override those in the
> >> unmodified_drivers.
> >>
> >> Likewise the process fails if you try to use a generic kernel source
> >> tree which has not been patched for Xen dom0, in which case the header
> >> files are missing various Xen declarations required for successful
> >> compilation.
> >>
> >> And here is a final proof, Xenified 2.6.31 with all XEN options
> >> disabled, with pv-on-hvm drivers:
> >>
> >> localhost ~ # uname -a
> >> Linux localhost 2.6.31-xen-r14 #2 SMP Mon Apr 12 13:34:27 GMT 2010
> >> x86_64 Intel(R) Xeon(R) CPU E5420 @ 2.50GHz GenuineIntel GNU/Linux
> >> localhost ~ # zgrep XEN /proc/config.gz
> >> # CONFIG_X86_64_XEN is not set
> >> localhost ~ # lsmod
> >> Module                  Size  Used by
> >> xen_vnif               28288  0
> >> xen_balloon            14496  1 xen_vnif
> >> xen_vbd                19084  3
> >> xen_platform_pci       78808  3 xen_vnif,xen_balloon,xen_vbd,[permanent]
> >>
> >> I'm not sure where to go next with this but I hope this info is useful.
> >>
> >
> > Congratulations!
> >
> > This is valuable information, thanks for going through that.
> >
> > Can you reproduce this with Xen 4.0.0 ? ie. does opensuse xen have
> > some patches related to pv-on-hvm drivers?
> 
> I believe SLES11SP1 will have Xen 4.0 but I have been unable to find
> anywhere to download srpms, apparently they are on dvd2 but as it is
> still in beta there are no iso images either, however opensuse factory
> does have a Xen 4.0 srpm:
> 
> http://download.opensuse.org/factory/repo/src-oss/suse/src/xen-4.0.0_20978_01-1.4.src.rpm
> 
> So the same process could be followed using that, I'm going to give it
> a try tomorrow.
> 

Ok.

> >
> > If there are some patches, those should be upstreamed to Xen.
> 
> Hmm, the patches may simply be there to allow building using the
> /usr/src/linux/include/xen headers instead of /usr/src/xen/ , and as
> xensource does not have a newer "classic" kernel than 2.6.18 why would
> they upsteam them.
> 

Actually there is nowadays.. :)

http://xenbits.xen.org/XCP/linux-2.6.27.pq.hg
(The kernel tree used by XCP/XenServer/XCI.)

-- Pasi

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Mon, Apr 12, 2010 at 05:05:37PM +0100, Andrew Lyon wrote:
> >>
> >> I'm not sure where to go next with this but I hope this info is useful.
> >>
> >
> > Congratulations!
> >
> > This is valuable information, thanks for going through that.
> >
> > Can you reproduce this with Xen 4.0.0 ? ie. does opensuse xen have
> > some patches related to pv-on-hvm drivers?
> 
> I believe SLES11SP1 will have Xen 4.0 but I have been unable to find
> anywhere to download srpms, apparently they are on dvd2 but as it is
> still in beta there are no iso images either, however opensuse factory
> does have a Xen 4.0 srpm:
> 
> http://download.opensuse.org/factory/repo/src-oss/suse/src/xen-4.0.0_20978_01-1.4.src.rpm
> 
> So the same process could be followed using that, I'm going to give it
> a try tomorrow.
> 

Did you have time to try this yet? 

> >
> > If there are some patches, those should be upstreamed to Xen.
> 
> Hmm, the patches may simply be there to allow building using the
> /usr/src/linux/include/xen headers instead of /usr/src/xen/ , and as
> xensource does not have a newer "classic" kernel than 2.6.18 why would
> they upsteam them.
> 

There has been a lot of demand for PV-on-HVM drivers recently so things
that make them easier to build should be integrated to Xen, imo.

Getting pvops-based pv-on-hvm drivers to upstream Linux is the ultimate goal of course :)

-- Pasi

Re: status of PV drivers in Xen 4.0

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

On Tue, 20 Apr 2010, Pasi Kärkkäinen wrote:
> Getting pvops-based pv-on-hvm drivers to upstream Linux is the ultimate goal of course :)
> 

I temporarily had to slow down my work on that because of other deadlines
I had in the last months, but soon I should be able to resume my work
full time.
I am currently trying to fix a save\restore bug that prevents pv
drivers or the domain from resuming correctly after a save\restore or a
migration.
In particular the call to dpm_suspend_start doesn't leave the kernel in
a good state in the PV on HVM case and I am not sure why: it seems to hang
half way through.
I need to call dpm_suspend_start because xenbus_dev_suspend has to be
called otherwise the pv drivers don't resume correctly.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Pasi Kärkkäinen]

On Tue, Apr 20, 2010 at 05:52:58PM +0100, Stefano Stabellini wrote:
> On Tue, 20 Apr 2010, Pasi Kärkkäinen wrote:
> > Getting pvops-based pv-on-hvm drivers to upstream Linux is the ultimate goal of course :)
> > 
> 
> I temporarily had to slow down my work on that because of other deadlines
> I had in the last months, but soon I should be able to resume my work
> full time.
> I am currently trying to fix a save\restore bug that prevents pv
> drivers or the domain from resuming correctly after a save\restore or a
> migration.
> In particular the call to dpm_suspend_start doesn't leave the kernel in
> a good state in the PV on HVM case and I am not sure why: it seems to hang
> half way through.
> I need to call dpm_suspend_start because xenbus_dev_suspend has to be
> called otherwise the pv drivers don't resume correctly.
>

Thanks for the update!

Hmm.. I guess the code in the old xenlinux based pv-on-hvm drivers is different enough
so that it cannot be compared? 

-- Pasi

Re: status of PV drivers in Xen 4.0

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1187 bytes --]

On Tue, 20 Apr 2010, Pasi Kärkkäinen wrote:
> On Tue, Apr 20, 2010 at 05:52:58PM +0100, Stefano Stabellini wrote:
> > On Tue, 20 Apr 2010, Pasi Kärkkäinen wrote:
> > > Getting pvops-based pv-on-hvm drivers to upstream Linux is the ultimate goal of course :)
> > > 
> > 
> > I temporarily had to slow down my work on that because of other deadlines
> > I had in the last months, but soon I should be able to resume my work
> > full time.
> > I am currently trying to fix a save\restore bug that prevents pv
> > drivers or the domain from resuming correctly after a save\restore or a
> > migration.
> > In particular the call to dpm_suspend_start doesn't leave the kernel in
> > a good state in the PV on HVM case and I am not sure why: it seems to hang
> > half way through.
> > I need to call dpm_suspend_start because xenbus_dev_suspend has to be
> > called otherwise the pv drivers don't resume correctly.
> >
> 
> Thanks for the update!
> 
> Hmm.. I guess the code in the old xenlinux based pv-on-hvm drivers is different enough
> so that it cannot be compared? 
> 
 

I am using the code under unmodified_drivers as a reference and that is
quite different.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: status of PV drivers in Xen 4.0

[Guillaume Rousse]

Le 12/04/2010 17:40, Andrew Lyon a écrit :
> On Mon, Apr 12, 2010 at 12:27 PM, Guillaume Rousse
> <Guillaume.Rousse@inria.fr> wrote:
>> Le 10/04/2010 14:58, Pasi Kärkkäinen a écrit :
>>>> Have you been successful building the drivers for .31 or .32?
>> I just tried 2.6.33 so far, without success.
>>
>>>> I'd like to make a set of patches available to apply to Xen 3.4.2 or
>>>> 4.0 to enable building the drivers with recent kernels and the forward
>>>> ported kernels I usually use, but while I've been able to rebuild them
>>>> on a opensuse 11.2 system running 2.6.31.12 I am finding it hard to
>>>> work out which patches need to be applied and what else the rpm build
>>>> process does to enable compilation.
>>>>
>>>> Any pointers would be much appreciated.
>>>>
>>>
>>> Did you take a look at novell's sles11 sp1 kernel? it's 2.6.32 and
>>> should have pv-on-hvm drivers included?
>> Yes, that's Jan Beulich said on this list. However, I've buid a
>> 2.6.32.11 kernel, with all xen patches available from SLE11-SP1 (as well
>> as a few others needed for applying xen patches properly), and used the
>> corresponding defconfig file. But I can't find any reference to those
>> drivers, at least with the names given in README file of xen tarball
>> (xen-platform-pci.ko, xen-vbd.ko, xen-vnif.ko and xen-balloon.ko).
>> That's probably just because of a name change.
>>
>> Anyway (unless I'm wrong), the actual usefulness of pv-on-hvm is on
>> standard linux kernels, not on dom0 itself, and the issue seems to be on
>> xen Makefile side.
>> --
> 
> I've been looking at this over the weekend and I've made some
> progress, you are correct about the usefulness of the pv-on-hvm
> drivers, however you need to bear in mind that the openSUSE/SLES11
> kernels *always* include the xen patches even if they are not
> activated for a particular kernel flavour (e.g. desktop, debug etc),
> so there are some differences in how the drivers are built,
> specifically the build process uses header files from the xenified
> kernel instead of from Xen itself.
OK, I understand the issue: you can't build those drivers against a
standard kernel, you need to have some xen patches applied, even if the
resulting options are  not activated.

Indeed, I have different build results when trying to build those
drivers against a xenified 2.6.32.11 kernel:
/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.c:
In function ‘balloon_init’:
/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
erreur: implicit declaration of function ‘pagevec_init’
/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
erreur: ‘free_pagevec’ undeclared (first use in this function)
/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
erreur: (Each undeclared identifier is reported only once
/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.c:583:
erreur: for each function it appears in.)
make[2]: ***
[/home/guillomovitch/cooker/xen/xen-4.0.0/unmodified_drivers/linux-2.6/balloon/balloon.o]
Erreur 1

Including "<linux/pagevec.h>" in ballon.c fixe the first error, but not
the second one.

Anyway, I can't ask the mandriva kernel team to include the whole xen
patches in all kernel just for this. I probably have to isolate the
strict subset of patches needed to make a kernel a suitable target for
those drivers.

-- 
BOFH excuse #223:

The lines are all busy (busied out, that is -- why let them in to begin
with?).