[PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

The following patch enables blktap and firmware using config
variables.
Specific issues are with POWERPC:
  1) does not require firmware build
  2) nasty build break in libaio

I'm guessing for IA64.

Signed-off-by: Jimi Xenidis <jimix@watson.ibm.com>

---
7476d config/ia64.mk
--- a/config/ia64.mk	Mon Aug 07 17:49:16 2006 -0500
+++ b/config/ia64.mk	Tue Aug 08 13:18:44 2006 -0400
@@ -1,5 +1,7 @@ CONFIG_IA64 := y
 CONFIG_IA64 := y
 CONFIG_IOEMU := y
 CONFIG_XCUTILS := y
+CONFIG_BLKTAP := y
+CONFIG_FIRMWARE := y
 
 LIBDIR := lib
diff -r 058f2e27476d config/x86_32.mk
--- a/config/x86_32.mk	Mon Aug 07 17:49:16 2006 -0500
+++ b/config/x86_32.mk	Tue Aug 08 13:18:44 2006 -0400
@@ -4,6 +4,8 @@ CONFIG_XCUTILS := y
 CONFIG_XCUTILS := y
 CONFIG_IOEMU := y
 CONFIG_MBOOTPACK := y
+CONFIG_BLKTAP := y
+CONFIG_FIRMWARE := y
 
 CFLAGS += -m32 -march=i686
 LIBDIR := lib
diff -r 058f2e27476d config/x86_64.mk
--- a/config/x86_64.mk	Mon Aug 07 17:49:16 2006 -0500
+++ b/config/x86_64.mk	Tue Aug 08 13:18:44 2006 -0400
@@ -4,6 +4,8 @@ CONFIG_XCUTILS := y
 CONFIG_XCUTILS := y
 CONFIG_IOEMU := y
 CONFIG_MBOOTPACK := y
+CONFIG_BLKTAP := y
+CONFIG_FIRMWARE := y
 
 CFLAGS += -m64
 LIBDIR = lib64
diff -r 058f2e27476d tools/Makefile
--- a/tools/Makefile	Mon Aug 07 17:49:16 2006 -0500
+++ b/tools/Makefile	Tue Aug 08 13:18:44 2006 -0400
@@ -1,4 +1,5 @@ XEN_ROOT = ../
 XEN_ROOT = ../
+include $(XEN_ROOT)/Config.mk
 include $(XEN_ROOT)/tools/Rules.mk
 
 SUBDIRS-y :=
@@ -8,16 +9,14 @@ SUBDIRS-y += examples
 SUBDIRS-y += examples
 SUBDIRS-y += xentrace
 SUBDIRS-$(CONFIG_XCUTILS) += xcutils
-SUBDIRS-y += firmware
+SUBDIRS-$(CONFIG_FIRMWARE) += firmware
 SUBDIRS-y += security
 SUBDIRS-y += console
 SUBDIRS-y += xenmon
 SUBDIRS-y += guest-headers
-SUBDIRS-$(VTPM_TOOLS) += vtpm_manager
-SUBDIRS-$(VTPM_TOOLS) += vtpm
+SUBDIRS-$(VTPM_TOOLS) += vtpm_manager vtpm
 SUBDIRS-y += xenstat
-SUBDIRS-y += libaio
-SUBDIRS-y += blktap
+SUBDIRS-$(CONFIG_BLKTAP) += libaio blktap
 
 # These don't cross-compile
 ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Alex Williamson]

On Tue, 2006-08-08 at 13:23 -0400, Jimi Xenidis wrote:
> The following patch enables blktap and firmware using config
> variables.
> Specific issues are with POWERPC:
>   1) does not require firmware build
>   2) nasty build break in libaio
> 
> I'm guessing for IA64.

   Looks ok to me.  We're not yet using blktap/libaio, but it builds.
Thanks,

	Alex

-- 
Alex Williamson                             HP Open Source & Linux Org.

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Keir Fraser]

On 8/8/06 6:23 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:

> The following patch enables blktap and firmware using config
> variables.
> Specific issues are with POWERPC:
>   1) does not require firmware build

The firmware is quite definitely x86-specific. May as well make the build
dependent on CONFIG_X86 rather than introduce another config variable.

>   2) nasty build break in libaio

Shouldn't you fix that instead of skirting the issue? :-)

 -- Keir

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

On Aug 9, 2006, at 5:41 AM, Keir Fraser wrote:

>
>
>
> On 8/8/06 6:23 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>
>> The following patch enables blktap and firmware using config
>> variables.
>> Specific issues are with POWERPC:
>>   1) does not require firmware build
>
> The firmware is quite definitely x86-specific. May as well make the  
> build
> dependent on CONFIG_X86 rather than introduce another config variable.
>
>>   2) nasty build break in libaio
>
> Shouldn't you fix that instead of skirting the issue? :-)
>
I knew you would say that! :)
Will you take patches to libaio, or are you expecting that to stay  
clean to the distro?
I may have missed this discussion, but why can't we just use the one  
that comes with the Distro?

-JX

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Keir Fraser]

On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:

>>>   2) nasty build break in libaio
>> 
>> Shouldn't you fix that instead of skirting the issue? :-)
>> 
> I knew you would say that! :)
> Will you take patches to libaio, or are you expecting that to stay
> clean to the distro?
> I may have missed this discussion, but why can't we just use the one
> that comes with the Distro?

That's a question for Julian (cc'ed). I assume it *ought* to work on
powerpc, so perhaps at worst we add a patch and also send it upstream.

 -- Keir

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>
>>>>   2) nasty build break in libaio
>>>
>>> Shouldn't you fix that instead of skirting the issue? :-)
>>>
>> I knew you would say that! :)
>> Will you take patches to libaio, or are you expecting that to stay
>> clean to the distro?
>> I may have missed this discussion, but why can't we just use the one
>> that comes with the Distro?
>
> That's a question for Julian (cc'ed). I assume it *ought* to work on
> powerpc, so perhaps at worst we add a patch and also send it upstream.

ok, I checked in the fix, you will get it in the next pull from our  
merge tree.
Here it is for reference:
changeset:   11432:e60de2425ec47caf50301b6aa164c61ff1540eb7
user:        Jimi Xenidis <jimix@watson.ibm.com>
date:        Wed Aug 09 15:38:37 2006 -0400
files:       tools/libaio/src/syscall-ppc.h
description:
[POWERPC][TOOLS] fix header so libaio builds

Signed-off-by: Jimi Xenidis <jimix@watson.ibm.com>


diff -r 0bb2e9a96c33777247319e6ff3428212bf5c1f17 -r  
e60de2425ec47caf50301b6aa164c61ff1540eb7 tools/libaio/src/syscall-ppc.h
--- a/tools/libaio/src/syscall-ppc.h	Wed Aug 09 15:37:42 2006 -0400
+++ b/tools/libaio/src/syscall-ppc.h	Wed Aug 09 15:38:37 2006 -0400
@@ -1,3 +1,6 @@
+#include <asm/unistd.h>
+#include <errno.h>
+
#define __NR_io_setup		227
#define __NR_io_destroy		228
#define __NR_io_getevents	229
@@ -9,7 +12,7 @@
   * "sc; bnslr" sequence) and CR (where only CR0.SO is clobbered to  
signal
   * an error return status).
   */
-
+#ifndef __syscall_nr
#define __syscall_nr(nr, type, name, args...)				\
	unsigned long __sc_ret, __sc_err;				\
	{								\
@@ -37,6 +40,7 @@
	}								\
	if (__sc_err & 0x10000000) return -((int)__sc_ret);		\
	return (type) __sc_ret
+#endif
#define __sc_loadargs_0(name, dummy...)					\
	__sc_0 = __NR_##name

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Julian Chesterfield]

Thanks Jim. I'll post the patch to the libaio folks also so we can keep 
things in synch.

- Julian

On 10 Aug 2006, at 02:06, Jimi Xenidis wrote:

>
> On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
>> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>>
>>>>>   2) nasty build break in libaio
>>>>
>>>> Shouldn't you fix that instead of skirting the issue? :-)
>>>>
>>> I knew you would say that! :)
>>> Will you take patches to libaio, or are you expecting that to stay
>>> clean to the distro?
>>> I may have missed this discussion, but why can't we just use the one
>>> that comes with the Distro?
>>
>> That's a question for Julian (cc'ed). I assume it *ought* to work on
>> powerpc, so perhaps at worst we add a patch and also send it upstream.
>
> ok, I checked in the fix, you will get it in the next pull from our 
> merge tree.
> Here it is for reference:
> changeset:   11432:e60de2425ec47caf50301b6aa164c61ff1540eb7
> user:        Jimi Xenidis <jimix@watson.ibm.com>
> date:        Wed Aug 09 15:38:37 2006 -0400
> files:       tools/libaio/src/syscall-ppc.h
> description:
> [POWERPC][TOOLS] fix header so libaio builds
>
> Signed-off-by: Jimi Xenidis <jimix@watson.ibm.com>
>
>
> diff -r 0bb2e9a96c33777247319e6ff3428212bf5c1f17 -r 
> e60de2425ec47caf50301b6aa164c61ff1540eb7 
> tools/libaio/src/syscall-ppc.h
> --- a/tools/libaio/src/syscall-ppc.h	Wed Aug 09 15:37:42 2006 -0400
> +++ b/tools/libaio/src/syscall-ppc.h	Wed Aug 09 15:38:37 2006 -0400
> @@ -1,3 +1,6 @@
> +#include <asm/unistd.h>
> +#include <errno.h>
> +
> #define __NR_io_setup		227
> #define __NR_io_destroy		228
> #define __NR_io_getevents	229
> @@ -9,7 +12,7 @@
>   * "sc; bnslr" sequence) and CR (where only CR0.SO is clobbered to 
> signal
>   * an error return status).
>   */
> -
> +#ifndef __syscall_nr
> #define __syscall_nr(nr, type, name, args...)				\
> 	unsigned long __sc_ret, __sc_err;				\
> 	{								\
> @@ -37,6 +40,7 @@
> 	}								\
> 	if (__sc_err & 0x10000000) return -((int)__sc_ret);		\
> 	return (type) __sc_ret
> +#endif
> #define __sc_loadargs_0(name, dummy...)					\
> 	__sc_0 = __NR_##name
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

On Aug 10, 2006, at 6:08 AM, Julian Chesterfield wrote:

> Thanks Jim. I'll post the patch to the libaio folks also so we can  
> keep things in synch.
>
> On 10 Aug 2006, at 02:06, Jimi Xenidis wrote:
>
>>
>> On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
>>> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>>>> I may have missed this discussion, but why can't we just use the  
>>>> one
>>>> that comes with the Distro?
>>>
>>> That's a question for Julian (cc'ed).


Thanks... but... you... never... said... why?

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Julian Chesterfield]

On 10 Aug 2006, at 11:51, Jimi Xenidis wrote:

>
> On Aug 10, 2006, at 6:08 AM, Julian Chesterfield wrote:
>
>> Thanks Jim. I'll post the patch to the libaio folks also so we can 
>> keep things in synch.
>>
>> On 10 Aug 2006, at 02:06, Jimi Xenidis wrote:
>>
>>>
>>> On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
>>>> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>>>>> I may have missed this discussion, but why can't we just use the 
>>>>> one
>>>>> that comes with the Distro?
>>>>
>>>> That's a question for Julian (cc'ed).
>
>
> Thanks... but... you... never... said... why?

Because libaio is often not included in the distro. In order to avoid 
breaking the build we either had to include checks in the Makefile for 
the existence of the library and build if necessary, or just statically 
build within the tree. We chose the latter since it was the cleanest 
approach.

- Julian

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

On Aug 10, 2006, at 9:51 AM, Julian Chesterfield wrote:

>
> On 10 Aug 2006, at 11:51, Jimi Xenidis wrote:
>
>>
>> On Aug 10, 2006, at 6:08 AM, Julian Chesterfield wrote:
>>
>>> Thanks Jim. I'll post the patch to the libaio folks also so we  
>>> can keep things in synch.
>>>
>>> On 10 Aug 2006, at 02:06, Jimi Xenidis wrote:
>>>
>>>>
>>>> On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
>>>>> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>>>>>> I may have missed this discussion, but why can't we just use  
>>>>>> the one
>>>>>> that comes with the Distro?
>>>>>
>>>>> That's a question for Julian (cc'ed).
>>
>>
>> Thanks... but... you... never... said... why?
>
> Because libaio is often not included in the distro.

You mean there are distros that do not have a libaio devel package?

> In order to avoid breaking the build we either had to include  
> checks in the Makefile for the existence of the library and build  
> if necessary, or just statically build within the tree.

Assuming the libaio devel package is available, how is this different  
than the recent build break that required me to install openssl-devel  
packages?

> We chose the latter since it was the cleanest approach.

warning: maintaining and monitoring an external package that your  
system depends on is always fraught with issues, but it your makework :(

-JX

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Julian Chesterfield]

On 10 Aug 2006, at 15:08, Jimi Xenidis wrote:

>
> On Aug 10, 2006, at 9:51 AM, Julian Chesterfield wrote:
>
>>
>> On 10 Aug 2006, at 11:51, Jimi Xenidis wrote:
>>
>>>
>>> On Aug 10, 2006, at 6:08 AM, Julian Chesterfield wrote:
>>>
>>>> Thanks Jim. I'll post the patch to the libaio folks also so we can 
>>>> keep things in synch.
>>>>
>>>> On 10 Aug 2006, at 02:06, Jimi Xenidis wrote:
>>>>
>>>>>
>>>>> On Aug 9, 2006, at 12:37 PM, Keir Fraser wrote:
>>>>>> On 9/8/06 1:34 pm, "Jimi Xenidis" <jimix@watson.ibm.com> wrote:
>>>>>>> I may have missed this discussion, but why can't we just use the 
>>>>>>> one
>>>>>>> that comes with the Distro?
>>>>>>
>>>>>> That's a question for Julian (cc'ed).
>>>
>>>
>>> Thanks... but... you... never... said... why?
>>
>> Because libaio is often not included in the distro.
>
> You mean there are distros that do not have a libaio devel package?

Correct.

>
>> In order to avoid breaking the build we either had to include checks 
>> in the Makefile for the existence of the library and build if 
>> necessary, or just statically build within the tree.
>
> Assuming the libaio devel package is available, how is this different 
> than the recent build break that required me to install openssl-devel 
> packages?

We link against libcrypto, however I wasn't aware that this requires 
the openssl-devel package. In my experience libcrypto typically ships 
as part of the base distro for utilities such as ssh, scp etc.. Perhaps 
this isn't the case?

>
>> We chose the latter since it was the cleanest approach.
>
> warning: maintaining and monitoring an external package that your 
> system depends on is always fraught with issues, but it your makework 
> :(

Certainly, we'd rather not maintain a separate version of libaio. Given 
the alternative options of either disabling the automatic blktap build 
or causing build failures for significant numbers of people this really 
seemed to be the only option. We're happy to revisit the Makefile build 
check option however if this is the preferred approach.

Thanks,
Julian

>
> -JX
>
>

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jeremy Katz]

On Thu, 2006-08-10 at 14:51 +0100, Julian Chesterfield wrote:
> Because libaio is often not included in the distro. In order to avoid 
> breaking the build we either had to include checks in the Makefile for 
> the existence of the library and build if necessary, or just statically 
> build within the tree. We chose the latter since it was the cleanest 
> approach.

Except that for distributions, if there's a bugfix which needs to be
applied to libaio, it now needs to go to two different places.  This is
a path that is all but guaranteed to be a headache for distros in the
future :(

Jeremy

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Anil Madhavapeddy]

On 10 Aug 2006, at 17:58, Jeremy Katz wrote:

> On Thu, 2006-08-10 at 14:51 +0100, Julian Chesterfield wrote:
>> Because libaio is often not included in the distro. In order to avoid
>> breaking the build we either had to include checks in the Makefile  
>> for
>> the existence of the library and build if necessary, or just  
>> statically
>> build within the tree. We chose the latter since it was the cleanest
>> approach.
>
> Except that for distributions, if there's a bugfix which needs to be
> applied to libaio, it now needs to go to two different places.   
> This is
> a path that is all but guaranteed to be a headache for distros in the
> future :(
>

Debian is the main culprit; the bug has been filed for a year about  
its broken libaio package, and no progress: http://bugs.debian.org/ 
cgi-bin/pkgreport.cgi?pkg=libaio

Specifically, bug #318795: http://bugs.debian.org/cgi-bin/ 
bugreport.cgi?bug=318795

I've CCed the Debian libaio maintainer (William Lee Irwin) in case he  
can comment on whether it'll be fixed soon or not.

-anil

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Sean Dague]

[-- Attachment #1.1: Type: text/plain, Size: 1226 bytes --]

On Thu, Aug 10, 2006 at 03:27:52PM +0100, Julian Chesterfield wrote:
<snip>
> >>Because libaio is often not included in the distro.
> >
> >You mean there are distros that do not have a libaio devel package?
> 
> Correct.
> 
> >
> >>In order to avoid breaking the build we either had to include checks 
> >>in the Makefile for the existence of the library and build if 
> >>necessary, or just statically build within the tree.
> >
> >Assuming the libaio devel package is available, how is this different 
> >than the recent build break that required me to install openssl-devel 
> >packages?
> 
> We link against libcrypto, however I wasn't aware that this requires 
> the openssl-devel package. In my experience libcrypto typically ships 
> as part of the base distro for utilities such as ssh, scp etc.. Perhaps 
> this isn't the case?

In more of the rpm and deb based distros, header files are seperate. 
Building against libcrypto (which comes with openssl), requires
openssl-devel (or openssl-dev in the debian case IIRC).

	-Sean

-- 
Sean Dague
IBM Linux Technology Center                     email: japh@us.ibm.com
Open Hypervisor Team                           alt: sldague@us.ibm.com

[-- Attachment #1.2: Type: application/pgp-signature, Size: 191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Sean Dague]

[-- Attachment #1.1: Type: text/plain, Size: 1062 bytes --]

On Thu, Aug 10, 2006 at 12:58:00PM -0400, Jeremy Katz wrote:
> On Thu, 2006-08-10 at 14:51 +0100, Julian Chesterfield wrote:
> > Because libaio is often not included in the distro. In order to avoid 
> > breaking the build we either had to include checks in the Makefile for 
> > the existence of the library and build if necessary, or just statically 
> > build within the tree. We chose the latter since it was the cleanest 
> > approach.
> 
> Except that for distributions, if there's a bugfix which needs to be
> applied to libaio, it now needs to go to two different places.  This is
> a path that is all but guaranteed to be a headache for distros in the
> future :(

Extremely concur.  All the extra security work that embedded libtiff and
libz caused over the past year or so should be plenty of reason to no embedd
a seperate copy of any library in a source tree.

	-Sean

-- 
Sean Dague
IBM Linux Technology Center                     email: japh@us.ibm.com
Open Hypervisor Team                           alt: sldague@us.ibm.com

[-- Attachment #1.2: Type: application/pgp-signature, Size: 191 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Christian Limpach]

On 8/10/06, Jeremy Katz <katzj@redhat.com> wrote:
> On Thu, 2006-08-10 at 14:51 +0100, Julian Chesterfield wrote:
> > Because libaio is often not included in the distro. In order to avoid
> > breaking the build we either had to include checks in the Makefile for
> > the existence of the library and build if necessary, or just statically
> > build within the tree. We chose the latter since it was the cleanest
> > approach.
>
> Except that for distributions, if there's a bugfix which needs to be
> applied to libaio, it now needs to go to two different places.  This is
> a path that is all but guaranteed to be a headache for distros in the
> future :(

Yes that's valid.  How about we add the proposed config option and set
its default from a test whether the required files exist?

    christian

Re: [PATCH][TOOLS] allow the selection of blktap and firmware thru config files

[Jimi Xenidis]

On Aug 11, 2006, at 7:03 AM, Christian Limpach wrote:

> On 8/10/06, Jeremy Katz <katzj@redhat.com> wrote:
>> On Thu, 2006-08-10 at 14:51 +0100, Julian Chesterfield wrote:
>> > Because libaio is often not included in the distro. In order to  
>> avoid
>> > breaking the build we either had to include checks in the  
>> Makefile for
>> > the existence of the library and build if necessary, or just  
>> statically
>> > build within the tree. We chose the latter since it was the  
>> cleanest
>> > approach.
>>
>> Except that for distributions, if there's a bugfix which needs to be
>> applied to libaio, it now needs to go to two different places.   
>> This is
>> a path that is all but guaranteed to be a headache for distros in the
>> future :(
>
> Yes that's valid.  How about we add the proposed config option and set
> its default from a test whether the required files exist?

hmm.. thinking about the test:
   for i in "/usr/lib/libaio.so.*"; do
     if [ -r /usr/lib/libaio.so ] ; then
       echo "please install libaio developer package, you are on  
debian please add the symlink"
       return fail
     else
       echo "using system provided library"
       return success
     fi
   done
   echo "please add libaio developer package"
   return fail

and now tools/libaio "disappears in a poof of logic" :)
no config option necessary.

Perhaps a handy build tools checker that makes sure that all  
everything you need to build and/or run is there.
we already check the version of python.

-JX