[Buildroot] Several issues with building multiple "projects" for same architecture

[Buildroot] Several issues with building multiple "projects" for same architecture

[Jonathan Nalley]

I am using the buildroot to build for several different boards, some of them
have the same CPU architecture (PPC405GP/EP).  I have noticed some issues
with the "project" concept in the buildroot.  Since the builds are for the
same architecture the toolchain is created under "build_powerpc".
Everything works fine for the first board, but for subsequent builds for
different "project" names things are missing under
"project_build_powerpc/PROJECT_NAME/root".  Specifically:

/root/lib/libgcc_s.so
/root/usr/lib/libstdc++.so
/root/usr/sbin/ethtool

This is because the make files that install those files check for:

$(GCC_BUILD_DIR2)/.libs_installed

and

$(ETHTOOL_DIR)/.installed

(There are several other packages whose make files also behave this way)

The problem with this is that the files are only installed for the first
project, subsequent projects see the .installed file in the BUILD directory
and think that files have already been copied to the TARGET directory.  Is
there a reason that the buildroot relies on .installed files instead of the
actual target file?

For example to fix ethtool I made the following change:

diff -r ff93bc269d3d buildroot/package/ethtool/ethtool.mk
--- a/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:03:23 2007
-0500
+++ b/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:50:36 2007
-0500
@@ -33,11 +33,11 @@ ETHTOOL_CAT:=$(ZCAT)
 $(ETHTOOL_DIR)/ethtool: $(ETHTOOL_DIR)/.configured
        $(MAKE) CC=$(TARGET_CC) -C $(ETHTOOL_DIR)

-$(ETHTOOL_DIR)/.installed: $(ETHTOOL_DIR)/ethtool
+$(TARGET_DIR)/usr/sbin/ethtool: $(ETHTOOL_DIR)/ethtool
        cp $(ETHTOOL_DIR)/ethtool $(TARGET_DIR)/usr/sbin
        touch $@

-ethtool: uclibc $(ETHTOOL_DIR)/.installed
+ethtool: uclibc $(TARGET_DIR)/usr/sbin/ethtool

 ethtool-source: $(DL_DIR)/$(ETHTOOL_SOURCE)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://busybox.net/lists/buildroot/attachments/20071029/04a8e654/attachment-0001.htm 

[Buildroot] Several issues with building multiple "projects" for same architecture

[Bernhard Fischer]

On Mon, Oct 29, 2007 at 10:55:49AM -0500, Jonathan Nalley wrote:
>I am using the buildroot to build for several different boards, some of them
>have the same CPU architecture (PPC405GP/EP).  I have noticed some issues
>with the "project" concept in the buildroot.  Since the builds are for the
>same architecture the toolchain is created under "build_powerpc".
>Everything works fine for the first board, but for subsequent builds for
>different "project" names things are missing under
>"project_build_powerpc/PROJECT_NAME/root".  Specifically:
>
>/root/lib/libgcc_s.so
>/root/usr/lib/libstdc++.so
>/root/usr/sbin/ethtool
>
>This is because the make files that install those files check for:
>
>$(GCC_BUILD_DIR2)/.libs_installed
>
>and
>
>$(ETHTOOL_DIR)/.installed

I've fixed at least the ethtool case several weeks ago in
f93f6bf4f9ae7c8c248fba2fc72ab42325975b63

As you can see, the project idea was never really integrated properly
:-/

[Buildroot] Several issues with building multiple "projects" forsame architecture

[Ulf Samuelsson]

From: "Jonathan Nalley" <buildroot@bluebot.org>
To: <buildroot@uclibc.org>
Sent: Monday, October 29, 2007 4:55 PM
Subject: [Buildroot] Several issues with building multiple "projects" forsame architecture


>I am using the buildroot to build for several different boards, some of them
> have the same CPU architecture (PPC405GP/EP).  I have noticed some issues
> with the "project" concept in the buildroot.  Since the builds are for the
> same architecture the toolchain is created under "build_powerpc".
> Everything works fine for the first board, but for subsequent builds for
> different "project" names things are missing under
> "project_build_powerpc/PROJECT_NAME/root".  Specifically:
> 
> /root/lib/libgcc_s.so
> /root/usr/lib/libstdc++.so
> /root/usr/sbin/ethtool
> 
> This is because the make files that install those files check for:
> 
> $(GCC_BUILD_DIR2)/.libs_installed
> 
> and
> 
> $(ETHTOOL_DIR)/.installed
> 
> (There are several other packages whose make files also behave this way)
> 
> The problem with this is that the files are only installed for the first
> project, subsequent projects see the .installed file in the BUILD directory
> and think that files have already been copied to the TARGET directory.  Is
> there a reason that the buildroot relies on .installed files instead of the
> actual target file?
> 
> For example to fix ethtool I made the following change:
> 
> diff -r ff93bc269d3d buildroot/package/ethtool/ethtool.mk
> --- a/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:03:23 2007
> -0500
> +++ b/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:50:36 2007
> -0500
> @@ -33,11 +33,11 @@ ETHTOOL_CAT:=$(ZCAT)
> $(ETHTOOL_DIR)/ethtool: $(ETHTOOL_DIR)/.configured
>        $(MAKE) CC=$(TARGET_CC) -C $(ETHTOOL_DIR)
> 
> -$(ETHTOOL_DIR)/.installed: $(ETHTOOL_DIR)/ethtool
> +$(TARGET_DIR)/usr/sbin/ethtool: $(ETHTOOL_DIR)/ethtool
>        cp $(ETHTOOL_DIR)/ethtool $(TARGET_DIR)/usr/sbin
>        touch $@
> 
> -ethtool: uclibc $(ETHTOOL_DIR)/.installed
> +ethtool: uclibc $(TARGET_DIR)/usr/sbin/ethtool
> 
> ethtool-source: $(DL_DIR)/$(ETHTOOL_SOURCE)


It might be better to define a dependencies directory,
$(DEPENDENCIES_DIR) = project_build_dir/$(PROJECT)/depend
so you can install the ".*' files in this directory.
In many places, you just do
make DESTDIR=xxx -C <package_dir> install


Best Regards
Ulf Samuelsson

[Buildroot] Several issues with building multiple "projects" for same architecture

[Ulf Samuelsson]

m?n 2007-10-29 klockan 10:55 -0500 skrev Jonathan Nalley:
> I am using the buildroot to build for several different boards, some
> of them have the same CPU architecture (PPC405GP/EP).  I have noticed
> some issues with the "project" concept in the buildroot.  Since the
> builds are for the same architecture the toolchain is created under
> "build_powerpc".  Everything works fine for the first board, but for
> subsequent builds for different "project" names things are missing
> under "project_build_powerpc/PROJECT_NAME/root".  Specifically: 
> 
> /root/lib/libgcc_s.so
> /root/usr/lib/libstdc++.so
> /root/usr/sbin/ethtool
> 
> This is because the make files that install those files check for:
> 
> $(GCC_BUILD_DIR2)/.libs_installed
> 
> and
> 
> $(ETHTOOL_DIR)/.installed 
> 
> (There are several other packages whose make files also behave this
> way)
> 
> The problem with this is that the files are only installed for the
> first project, subsequent projects see the .installed file in the
> BUILD directory and think that files have already been copied to the
> TARGET directory.  Is there a reason that the buildroot relies
> on .installed files instead of the actual target file? 
> 
> For example to fix ethtool I made the following change:
> 
> diff -r ff93bc269d3d buildroot/package/ethtool/ethtool.mk
> --- a/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:03:23
> 2007 -0500
> +++ b/buildroot/package/ethtool/ethtool.mk      Mon Oct 29 10:50:36
> 2007 -0500 
> @@ -33,11 +33,11 @@ ETHTOOL_CAT:=$(ZCAT)
>  $(ETHTOOL_DIR)/ethtool: $(ETHTOOL_DIR)/.configured
>         $(MAKE) CC=$(TARGET_CC) -C $(ETHTOOL_DIR)
>  
> -$(ETHTOOL_DIR)/.installed: $(ETHTOOL_DIR)/ethtool
> +$(TARGET_DIR)/usr/sbin/ethtool: $(ETHTOOL_DIR)/ethtool 
>         cp $(ETHTOOL_DIR)/ethtool $(TARGET_DIR)/usr/sbin
>         touch $@
>  
> -ethtool: uclibc $(ETHTOOL_DIR)/.installed
> +ethtool: uclibc $(TARGET_DIR)/usr/sbin/ethtool
>  
>  ethtool-source: $(DL_DIR)/$(ETHTOOL_SOURCE) 
> _______________________________________________
> buildroot mailing list
> buildroot at uclibc.org
> http://busybox.net/mailman/listinfo/buildroot

Here is a first attempt to fix this,
I have not tested it yet myself though.

The idea is to create a $(DEP_DIR) directory in $(PROJECT_BUILD_DIR)
and then to generate a $(<package>).installed in this directory.


-- 
Best Regards
Ulf Samuelsson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: target_install.patch
Type: text/x-patch
Size: 17526 bytes
Desc: not available
Url : http://busybox.net/lists/buildroot/attachments/20071029/cfd057e9/attachment.bin 

[Buildroot] Several issues with building multiple "projects" for same architecture

[Hamish Moffatt]

On Mon, Oct 29, 2007 at 11:50:07PM +0100, Ulf Samuelsson wrote:
> m?n 2007-10-29 klockan 10:55 -0500 skrev Jonathan Nalley:
> > I am using the buildroot to build for several different boards, some
> > of them have the same CPU architecture (PPC405GP/EP).  I have noticed
> > some issues with the "project" concept in the buildroot.  Since the
> > builds are for the same architecture the toolchain is created under
> > "build_powerpc".  Everything works fine for the first board, but for
> > subsequent builds for different "project" names things are missing
> > under "project_build_powerpc/PROJECT_NAME/root".  Specifically: 
> > 
> > /root/lib/libgcc_s.so
> > /root/usr/lib/libstdc++.so
> > /root/usr/sbin/ethtool
> > 
> > This is because the make files that install those files check for:
> > 
> > $(GCC_BUILD_DIR2)/.libs_installed

There's another problem with this approach. If you are modifying the
target_skeleton contents you might want to nuke the generated root
directory and have it rebuilt, but then libgcc_s.so (etc) won't be
re-copied.

> Here is a first attempt to fix this,
> I have not tested it yet myself though.
> 
> The idea is to create a $(DEP_DIR) directory in $(PROJECT_BUILD_DIR)
> and then to generate a $(<package>).installed in this directory.

This doesn't solve the problem I note either, although I guess you could
also delete *.installed. Why not just depend on the actual target rather
than these stamp files?


thanks,
Hamish
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

[Buildroot] Several issues with building multiple"projects" for same architecture

[Ulf Samuelsson]

On Mon, Oct 29, 2007 at 11:50:07PM +0100, Ulf Samuelsson wrote:
> m?n 2007-10-29 klockan 10:55 -0500 skrev Jonathan Nalley:
> > I am using the buildroot to build for several different boards, some
> > of them have the same CPU architecture (PPC405GP/EP).  I have noticed
> > some issues with the "project" concept in the buildroot.  Since the
> > builds are for the same architecture the toolchain is created under
> > "build_powerpc".  Everything works fine for the first board, but for
> > subsequent builds for different "project" names things are missing
> > under "project_build_powerpc/PROJECT_NAME/root".  Specifically:
> >
> > /root/lib/libgcc_s.so
> > /root/usr/lib/libstdc++.so
> > /root/usr/sbin/ethtool
> >
> > This is because the make files that install those files check for:
> >
> > $(GCC_BUILD_DIR2)/.libs_installed

There's another problem with this approach. If you are modifying the
target_skeleton contents you might want to nuke the generated root
directory and have it rebuilt, but then libgcc_s.so (etc) won't be
re-copied.

> Here is a first attempt to fix this,
> I have not tested it yet myself though.
>
> The idea is to create a $(DEP_DIR) directory in $(PROJECT_BUILD_DIR)
> and then to generate a $(<package>).installed in this directory.

This doesn't solve the problem I note either, although I guess you could
also delete *.installed. Why not just depend on the actual target rather
than these stamp files?

==> The proper way of installing a package is often "make install".
        Question is which files should be copied and in which order.

        If you depend on an arbitrary file, then you risk that
        the build fails if not all files are copied, but the file,
         on which we depend *is* copied

        If you detect the order and make it depend on the latest
        then you have a problem if the package is updated
        and the order is changed or if new files are introduced
        by make install.

        You also have a problem if the package has options.

        One possible option is to create the .installed
        inside the root file ssytem, but then you add unneccessary
        files to your target system.

        If you want to rebuild the project from scratch, then you
        might as well delete the complete $(PROJECT_BUILD_DIR)
        instead of the root file system.

        Note that my fix does not handle the "package/Makefile.in.autotools"
        Someone obviously needs to fix that as well.

Best Regards
Ulf Samuelsson