[PATCH] minios: minor buildsystem fixes

[PATCH] minios: minor buildsystem fixes

[Christoph Egger]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]


Hi!

Attached patch changes MiniOS.mk and tools/Rules.mk
use the OS settings we build on.

Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>

-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

[-- Attachment #2: xen_minios.diff --]
[-- Type: text/x-diff, Size: 950 bytes --]

diff -r 323ae92f774b config/MiniOS.mk
--- a/config/MiniOS.mk	Wed Jul 15 09:14:19 2009 +0100
+++ b/config/MiniOS.mk	Wed Jul 15 11:35:03 2009 +0200
@@ -1,4 +1,4 @@
-include $(XEN_ROOT)/config/StdGNU.mk
+include $(XEN_ROOT)/config/$(shell uname -s).mk
 include $(XEN_ROOT)/extras/mini-os/Config.mk
 CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
 CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)
diff -r 323ae92f774b tools/Rules.mk
--- a/tools/Rules.mk	Wed Jul 15 09:14:19 2009 +0100
+++ b/tools/Rules.mk	Wed Jul 15 11:35:03 2009 +0200
@@ -33,7 +33,10 @@ CFLAGS += -D__XEN_TOOLS__
 CFLAGS += -MMD -MF .$(@F).d
 DEPS = .*.d
 
-ifneq ($(XEN_OS),NetBSD)
+# On NetBSD LFS is always on.
+# We can't use $(XEN_OS) or we will miss MiniOS
+# when we build MiniOS on NetBSD.
+ifneq ($(shell uname -s),NetBSD)
 # Enable implicit LFS support *and* explicit LFS names.
 CFLAGS  += $(shell getconf LFS_CFLAGS)
 CFLAGS  += -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] minios: minor buildsystem fixes

[Keir Fraser]

On 15/07/2009 10:42, "Christoph Egger" <Christoph.Egger@amd.com> wrote:

> Attached patch changes MiniOS.mk and tools/Rules.mk
> use the OS settings we build on.
> 
> Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>

This may need some thought. Using XEN_OS in some places and open coding
'uname -s' in others is just not nice and is fragile.

Since stubdom is not being built in its own chroot, does it make sense for
it to override XEN_OS at all? Should we keep that as 'build host OS'? The
change around LFS in tools/Rules.mk could be indicative of a deeper issue --
information about arbitrary (non-GNU) host environments leaking into
binaries which are actually targetting GNUish minios. Can the current
approach work safely for NetBSD, Solaris, ... build hosts?

So I throw this one to Stefano/Samuel. ;-)

 -- Keir

Re: [PATCH] minios: minor buildsystem fixes

[Christoph Egger]

On Wednesday 15 July 2009 12:40:22 Keir Fraser wrote:
> On 15/07/2009 10:42, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
> > Attached patch changes MiniOS.mk and tools/Rules.mk
> > use the OS settings we build on.
> >
> > Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>
>
> This may need some thought. Using XEN_OS in some places and open coding
> 'uname -s' in others is just not nice and is fragile.
>
> Since stubdom is not being built in its own chroot, does it make sense for
> it to override XEN_OS at all? Should we keep that as 'build host OS'? The
> change around LFS in tools/Rules.mk could be indicative of a deeper issue
> -- information about arbitrary (non-GNU) host environments leaking into
> binaries which are actually targetting GNUish minios. Can the current
> approach work safely for NetBSD, Solaris, ... build hosts?
>
> So I throw this one to Stefano/Samuel. ;-)

The current approach works for Linux only.
NetBSD's base system is designed for cross-building.
My roadmap is to change the way how to build mini-os/stubdom
fundamentally:

The new infrastructure I have in mind will allow to choose if you use
newlib, libz, lwip and pciutils or use the native libs from the
base system.

Linux build will default to first, NetBSD build will default to latter one.

NetBSD libc doesn't have linux integer types, that's why mini-os
has been moved to C99 integer types. The mini-os headers need
some re-structuring to not conflict with the native libc, namely
if you want a mini-os header you explicitely have to type
<mini-os/header.h>

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [PATCH] minios: minor buildsystem fixes

[Stefano Stabellini]

On Wed, 15 Jul 2009, Christoph Egger wrote:
> The current approach works for Linux only.
> NetBSD's base system is designed for cross-building.
> My roadmap is to change the way how to build mini-os/stubdom
> fundamentally:
> 
> The new infrastructure I have in mind will allow to choose if you use
> newlib, libz, lwip and pciutils or use the native libs from the
> base system.
> 
> Linux build will default to first, NetBSD build will default to latter one.

I like the idea, could you please explain with a little bit more details
how the crossbuilding works on NetBSD?
Would it be possible to use NetBSD libc instead of newlib even on Linux?

However Keir is right about XEN_OS: the stubdom Makefile
shouldn't change it but should only set CONFIG_MiniOS instead.

Re: [PATCH] minios: minor buildsystem fixes

[Keir Fraser]

On 15/07/2009 13:35, "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>
wrote:

>> The new infrastructure I have in mind will allow to choose if you use
>> newlib, libz, lwip and pciutils or use the native libs from the
>> base system.
>> 
>> Linux build will default to first, NetBSD build will default to latter one.
> 
> I like the idea, could you please explain with a little bit more details
> how the crossbuilding works on NetBSD?
> Would it be possible to use NetBSD libc instead of newlib even on Linux?
> 
> However Keir is right about XEN_OS: the stubdom Makefile
> shouldn't change it but should only set CONFIG_MiniOS instead.

If the distinction that Christoph's patch introduced *were* to be required,
I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably
it would be clearer to rename existing XEN_OS to the former anyway?

 -- Keir

Re: [PATCH] minios: minor buildsystem fixes

[Stefano Stabellini]

On Wed, 15 Jul 2009, Keir Fraser wrote:
> On 15/07/2009 13:35, "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>
> wrote:
> 
> >> The new infrastructure I have in mind will allow to choose if you use
> >> newlib, libz, lwip and pciutils or use the native libs from the
> >> base system.
> >> 
> >> Linux build will default to first, NetBSD build will default to latter one.
> > 
> > I like the idea, could you please explain with a little bit more details
> > how the crossbuilding works on NetBSD?
> > Would it be possible to use NetBSD libc instead of newlib even on Linux?
> > 
> > However Keir is right about XEN_OS: the stubdom Makefile
> > shouldn't change it but should only set CONFIG_MiniOS instead.
> 
> If the distinction that Christoph's patch introduced *were* to be required,
> I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably
> it would be clearer to rename existing XEN_OS to the former anyway?
> 

I agree on that, XEN_OS as it is now is misleading.

Re: [PATCH] minios: minor buildsystem fixes

[Keir Fraser]

On 15/07/2009 13:43, "Stefano Stabellini" <Stefano.Stabellini@eu.citrix.com>
wrote:

>> If the distinction that Christoph's patch introduced *were* to be required,
>> I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably
>> it would be clearer to rename existing XEN_OS to the former anyway?
>> 
> 
> I agree on that, XEN_OS as it is now is misleading.

I changed my mind after all that. Obviously it is rather entrenched in
tools/ right now that the compile OS is also the target OS. Renaming XEN_OS
to one or the other gives the misleading impression we can cross-compile
across operating systems, which we can't.

I think your plan to leave XEN_OS alone and just define CONFIG_MiniOS -- and
then hack makefiles a bit where necessary to explicitly take note of the
latter -- makes most sense.

 -- Keir

Re: [PATCH] minios: minor buildsystem fixes

[Christoph Egger]

On Wednesday 15 July 2009 14:35:37 Stefano Stabellini wrote:
> On Wed, 15 Jul 2009, Christoph Egger wrote:
> > The current approach works for Linux only.
> > NetBSD's base system is designed for cross-building.
> > My roadmap is to change the way how to build mini-os/stubdom
> > fundamentally:
> >
> > The new infrastructure I have in mind will allow to choose if you use
> > newlib, libz, lwip and pciutils or use the native libs from the
> > base system.
> >
> > Linux build will default to first, NetBSD build will default to latter
> > one.
>
> I like the idea, could you please explain with a little bit more details
> how the crossbuilding works on NetBSD?

The headers are in /usr/include. The arch-dependent headers are in
/usr/include/<arch>/. These are automatically included by the standard
headers.

The native libs are in /lib and in /usr/lib/. The libs for the other 
architectures are in /usr/lib/<arch>/.

On x86, it is enough to specify -m64 or -m32 to tell gcc and its linker
if you want to build a binary for amd64 or i386.

If you want to build for other archs, you build a toolchain first which is
very easy with the build.sh script.


> Would it be possible to use NetBSD libc instead of newlib even on Linux?

It's not just to take the libc sources. You need the whole infrastructure
around it which would let you very likely end up in a half new linux 
distribution. I don't think, this is what you want. :)

BTW: I don't mind if you want to port Xen to a VAX :-D

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [PATCH] minios: minor buildsystem fixes

[Ian Jackson]

Christoph Egger writes ("[Xen-devel] [PATCH] minios: minor buildsystem fixes"):
> Attached patch changes MiniOS.mk and tools/Rules.mk
> use the OS settings we build on.

I don't think that's right.  The minios cross-build should use the Xen
cross build machinery.  That machinery isn't very good but it is the
same on every host.

Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem fixes"):
> The current approach works for Linux only.

Really ?  What's wrong with it for NetBSD ?

> NetBSD's base system is designed for cross-building.

I don't think we should be trying to use the host operating system's
cross-build support.

> The new infrastructure I have in mind will allow to choose if you use
> newlib, libz, lwip and pciutils or use the native libs from the
> base system.

And certainly not the host operating system's libraries for our
cross-build !

> Linux build will default to first, NetBSD build will default to latter one.

In particular I think this is quite wrong.

> NetBSD libc doesn't have linux integer types, that's why mini-os
> has been moved to C99 integer types. The mini-os headers need
> some re-structuring to not conflict with the native libc, namely
> if you want a mini-os header you explicitely have to type
> <mini-os/header.h>

I agree that it would be good to replace the minios libc.  The current
libc is horrid.  The BSD libc is very nice (and I have found it a joy
to port to a new embedded target, in the past).

But if we change to a new libc this should be done by always using the
new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another
choice) even on Linux hosts.

Ian.

Re: [PATCH] minios: minor buildsystem fixes

[Stefano Stabellini]

On Wed, 15 Jul 2009, Ian Jackson wrote:
> > Linux build will default to first, NetBSD build will default to latter one.
> 
> In particular I think this is quite wrong.

After giving it more thoughts, I agree with Ian.

> 
> > NetBSD libc doesn't have linux integer types, that's why mini-os
> > has been moved to C99 integer types. The mini-os headers need
> > some re-structuring to not conflict with the native libc, namely
> > if you want a mini-os header you explicitely have to type
> > <mini-os/header.h>
> 
> I agree that it would be good to replace the minios libc.  The current
> libc is horrid.  The BSD libc is very nice (and I have found it a joy
> to port to a new embedded target, in the past).
> 
> But if we change to a new libc this should be done by always using the
> new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another
> choice) even on Linux hosts.
>

In the process of making qemu stubdoms work, we certainly found many
shortcomings in newlib, so I welcome the idea of switching to a different
libc, especially if we want to make minios a good environment to run other
applications.
But maintaining a cross-compile environment is hard enough, we don't
want to maintain two of them, so the new libc should compile on Linux
hosts too.

Re: [PATCH] minios: minor buildsystem fixes

[Samuel Thibault]

Hello,

Ian Jackson, le Wed 15 Jul 2009 17:03:36 +0100, a écrit :
> > The new infrastructure I have in mind will allow to choose if you use
> > newlib, libz, lwip and pciutils or use the native libs from the
> > base system.
> 
> And certainly not the host operating system's libraries for our
> cross-build !

Yes, that's asking for problems.

> > NetBSD libc doesn't have linux integer types, that's why mini-os
> > has been moved to C99 integer types. The mini-os headers need
> > some re-structuring to not conflict with the native libc, namely
> > if you want a mini-os header you explicitely have to type
> > <mini-os/header.h>
> 
> I agree that it would be good to replace the minios libc.  The current
> libc is horrid.  The BSD libc is very nice (and I have found it a joy
> to port to a new embedded target, in the past).

That should be fine yes.  I don't remember precisely the reasons why I
chose newlib.  I didn't know the BSD libc had embedded targets too.

> But if we change to a new libc this should be done by always using the
> new libc even on Linux hosts.

Yes.  We should avoid having to fix several libcs.

Samuel

Re: [PATCH] minios: minor buildsystem fixes

[Christoph Egger]

On Wednesday 15 July 2009 18:03:36 Ian Jackson wrote:
> Christoph Egger writes ("[Xen-devel] [PATCH] minios: minor buildsystem 
fixes"):
> > Attached patch changes MiniOS.mk and tools/Rules.mk
> > use the OS settings we build on.
>
> I don't think that's right.  The minios cross-build should use the Xen
> cross build machinery.  That machinery isn't very good but it is the
> same on every host.

It's not the same on every host due to lack of proper abstraction.


> Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem 
fixes"):
> > The current approach works for Linux only.
>
> Really ?  What's wrong with it for NetBSD ?

Assumptions which don't apply to NetBSD.

> > NetBSD's base system is designed for cross-building.
>
> I don't think we should be trying to use the host operating system's
> cross-build support.

Please explain why not.
It's a question of having a proper abstraction.

> > The new infrastructure I have in mind will allow to choose if you use
> > newlib, libz, lwip and pciutils or use the native libs from the
> > base system.
>
> And certainly not the host operating system's libraries for our
> cross-build !

Please explain why not.
It's a question of having a proper abstraction.

> > Linux build will default to first, NetBSD build will default to latter
> > one.
>
> In particular I think this is quite wrong.

Please explain why.

> > NetBSD libc doesn't have linux integer types, that's why mini-os
> > has been moved to C99 integer types. The mini-os headers need
> > some re-structuring to not conflict with the native libc, namely
> > if you want a mini-os header you explicitely have to type
> > <mini-os/header.h>
>
> I agree that it would be good to replace the minios libc.  The current
> libc is horrid.  The BSD libc is very nice (and I have found it a joy
> to port to a new embedded target, in the past).
>
> But if we change to a new libc this should be done by always using the
> new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another
> choice) even on Linux hosts.

I think, let's first find a proper abstraction and then make the decision what
is best for which host.

Christoph

-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [PATCH] minios: minor buildsystem fixes

[Samuel Thibault]

Christoph Egger, le Thu 16 Jul 2009 14:17:20 +0200, a écrit :
> > And certainly not the host operating system's libraries for our
> > cross-build !
> 
> Please explain why not.
> It's a question of having a proper abstraction.

It's a problem of ABI. For instance, there is no way to implement
x86_64's red zone without adding a kernel/user separation in MiniOS ;
libcs usually assume that the red zone is available, thus some patching
to newlib for instance.

> > > Linux build will default to first, NetBSD build will default to latter
> > > one.
> >
> > In particular I think this is quite wrong.
> 
> Please explain why.

It's asking for having to fix issues coming from both linux' case and
NetBSD's case, while using the same libc all the time would bring just
one case.

Samuel

Re: [PATCH] minios: minor buildsystem fixes

[Ian Jackson]

Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem fixes"):
> On Wednesday 15 July 2009 18:03:36 Ian Jackson wrote:
> > Really ?  What's wrong with it for NetBSD ?
> 
> Assumptions which don't apply to NetBSD.

Specifically.

> > > NetBSD's base system is designed for cross-building.
> >
> > I don't think we should be trying to use the host operating system's
> > cross-build support.
> 
> Please explain why not.
> It's a question of having a proper abstraction.

No, it's a question of trying to run two parallel cross build
arrangements.

> I think, let's first find a proper abstraction and then make the
> decision what is best for which host.

It does not make sense that the results of a cross-build (such as
which libc is used) should depend on the host.

Ian.

Re: [PATCH] minios: minor buildsystem fixes

[Stefano Stabellini]

On Wed, 15 Jul 2009, Christoph Egger wrote:
> 
> Hi!
> 
> Attached patch changes MiniOS.mk and tools/Rules.mk
> use the OS settings we build on.
> 

Looking at the original patch of this thread, I realize that probably
none of this is actually needed: 

> diff -r 323ae92f774b config/MiniOS.mk
> --- a/config/MiniOS.mk	Wed Jul 15 09:14:19 2009 +0100
> +++ b/config/MiniOS.mk	Wed Jul 15 11:35:03 2009 +0200
> @@ -1,4 +1,4 @@
> -include $(XEN_ROOT)/config/StdGNU.mk
> +include $(XEN_ROOT)/config/$(shell uname -s).mk
>  include $(XEN_ROOT)/extras/mini-os/Config.mk
>  CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
>  CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)

This is not needed because the only things that NetBSD.mk adds more than
StdGNU.mk are not needed in the stubdom case.


> diff -r 323ae92f774b tools/Rules.mk
> --- a/tools/Rules.mk	Wed Jul 15 09:14:19 2009 +0100
> +++ b/tools/Rules.mk	Wed Jul 15 11:35:03 2009 +0200
> @@ -33,7 +33,10 @@ CFLAGS += -D__XEN_TOOLS__
>  CFLAGS += -MMD -MF .$(@F).d
>  DEPS = .*.d
>  
> -ifneq ($(XEN_OS),NetBSD)
> +# On NetBSD LFS is always on.
> +# We can't use $(XEN_OS) or we will miss MiniOS
> +# when we build MiniOS on NetBSD.
> +ifneq ($(shell uname -s),NetBSD)
>  # Enable implicit LFS support *and* explicit LFS names.
>  CFLAGS  += $(shell getconf LFS_CFLAGS)
>  CFLAGS  += -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE

Given that you are still using GNU gcc on NetBSD, even if LFS is already
enabled by default, how is this an error?
Does it actually prevent you from compiling successfully qemu?

If you really need to disable those flags you should do that based on
getconf responses.