[Buildroot] [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory

[Buildroot] [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory

[MidCheck]

When built with the system's toolchain, the output is as follows:
>>> toolchain-external-custom  Installing to target
>>> toolchain-external-custom  Copying external toolchain libraries to target...
make: *** [package/pkg-generic.mk:384: /root/buildroot-2022.02.11/output/build/toolchain-external-custom/.stamp_target_installed] Error 255

After echo the value of LIBPATH, I found it is a directory:
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d

So it caused the script to execute the "exit -1". When I added the processing of the directory, it was successfully built.

Signed-off-by: MidCheck <mc.xin@foxmail.com>
---
 toolchain/helpers.mk | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 24c482923a..e0ab991bec 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -28,6 +28,10 @@ copy_toolchain_lib_root = \
 			elif test -f $${LIBPATH}; then \
 				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 				break ; \
+			elif test -d $${LIBPATH}; then \
+				mkdir -p $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
+				cp -r $${LIBPATH}/* $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
+				break ; \
 			else \
 				exit -1; \
 			fi; \
-- 
2.40.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory

[Yann E. MORIN]

MidCheck, All,

On 2023-04-23 23:54 +0800, MidCheck spake thusly:
> When built with the system's toolchain, the output is as follows:
> >>> toolchain-external-custom  Installing to target
> >>> toolchain-external-custom  Copying external toolchain libraries to target...
> make: *** [package/pkg-generic.mk:384: /root/buildroot-2022.02.11/output/build/toolchain-external-custom/.stamp_target_installed] Error 255
> 
> After echo the value of LIBPATH, I found it is a directory:
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d

This is weird that you get an ld.so.conf.d in your toolchain. Is it a
toolchain we can have access to? How did it get built?

> So it caused the script to execute the "exit -1". When I added the processing of the directory, it was successfully built.
> 
> Signed-off-by: MidCheck <mc.xin@foxmail.com>

Please, use your real name, not a nickname. Note that it must also match
the authorship. Non-latin-script characters are perfectly OK.

> ---
>  toolchain/helpers.mk | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index 24c482923a..e0ab991bec 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -28,6 +28,10 @@ copy_toolchain_lib_root = \
>  			elif test -f $${LIBPATH}; then \
>  				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
>  				break ; \
> +			elif test -d $${LIBPATH}; then \
> +				mkdir -p $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
> +				cp -r $${LIBPATH}/* $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
> +				break ; \

The proper fix would be to exclude directories in the find on line 13.

Indeed, LIBPATTERN is the argument to copy_toolchain_lib_root, and in
the case of external toolchains, is each item in TOOLCHAIN_EXTERNAL_LIBS,
the first of which is ld*.so.*, which we expect to only match files, but
in your case matches a directory.

As I understand it, ld.so.conf is not used at build time to find
libraries, so having it in the staging is useless.

Also note that having /etc/ld.so.conf.d in the target is not supported,
and that Buildroot will explicitly fail at the end of the build if it
sees such a directory, as it is not possible to have proper ldconfig
setup in cross-compilation; see the top-level Makefile, lines 753-756,
and commit 9c4072348960 (Makefile: drop ldconfig handling). And so it is
even weirder that your change makes the build succeed...

Regards,
Yann E. MORIN.

>  			else \
>  				exit -1; \
>  			fi; \
> -- 
> 2.40.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] =?gb18030?b?u9i4tKO6ICBbUEFUQ0ggMS8xXSB0b29sY2hhaW4v?= =?gb18030?q?helpers=2Emk=3A_add_the_processing_when_LIBPATH_is_a_directory?=

[=?gb18030?B?SmlhbmdYaW4=?=]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: Type: text/plain; charset="gb18030", Size: 5995 bytes --]

Yann E. MORIN, ALL,


I build rootfs in docker container based on ubuntu22.04, host arch is x86_64.
I chose the gcc downloaded by apt to build. 


After the shell command in helpers.mk is excuted, the following content is output:
root@824b1b82dfae:~/buildroot-2022.02.11# find /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot -name "ld*.so.*"
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.cache
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/ld-linux-x86-64.so.2
/root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/lib/ld-linux-x86-64.so.2
root@824b1b82dfae:~/buildroot-2022.02.11# ls -l /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d
total 12
-rw-r--r-- 1 root root  38 Mar  5  2022 fakeroot-x86_64-linux-gnu.conf
-rw-r--r-- 1 root root  44 Sep 24  2021 libc.conf
-rw-r--r-- 1 root root 100 Sep 28  2021 x86_64-linux-gnu.conf



I added the echo on lines 753-756 of the top-level Makefile, and the build 
log shows it doesn't get there. Should the processing of directories be added
in toolchain/helpers.mk?


Finally, I changed name to my real name: JiangXin. Thank you very much
for pointint this out.


Regards,
JiangXin.


------------------ Ô­Ê¼Óʼþ ------------------
·¢¼þÈË:                                                                                                                        "Yann E. MORIN"                                                                                    <yann.morin.1998@free.fr&gt;;
·¢ËÍʱ¼ä:&nbsp;2023Äê4ÔÂ24ÈÕ(ÐÇÆÚÒ») Á賿0:40
ÊÕ¼þÈË:&nbsp;"JiangXin"<mc.xin@foxmail.com&gt;;
³­ËÍ:&nbsp;"buildroot"<buildroot@buildroot.org&gt;;"Romain Naour"<romain.naour@gmail.com&gt;;"Giulio Benetti"<giulio.benetti@benettiengineering.com&gt;;"Thomas De Schampheleire"<thomas.de_schampheleire@nokia.com&gt;;"Thomas Petazzoni"<thomas.petazzoni@bootlin.com&gt;;
Ö÷Ìâ:&nbsp;Re: [Buildroot] [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory



MidCheck, All,

On 2023-04-23 23:54 +0800, MidCheck spake thusly:
&gt; When built with the system's toolchain, the output is as follows:
&gt; &gt;&gt;&gt; toolchain-external-custom&nbsp; Installing to target
&gt; &gt;&gt;&gt; toolchain-external-custom&nbsp; Copying external toolchain libraries to target...
&gt; make: *** [package/pkg-generic.mk:384: /root/buildroot-2022.02.11/output/build/toolchain-external-custom/.stamp_target_installed] Error 255
&gt; 
&gt; After echo the value of LIBPATH, I found it is a directory:
&gt; /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d

This is weird that you get an ld.so.conf.d in your toolchain. Is it a
toolchain we can have access to? How did it get built?

&gt; So it caused the script to execute the "exit -1". When I added the processing of the directory, it was successfully built.
&gt; 
&gt; Signed-off-by: MidCheck <mc.xin@foxmail.com&gt;

Please, use your real name, not a nickname. Note that it must also match
the authorship. Non-latin-script characters are perfectly OK.

&gt; ---
&gt;&nbsp; toolchain/helpers.mk | 4 ++++
&gt;&nbsp; 1 file changed, 4 insertions(+)
&gt; 
&gt; diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
&gt; index 24c482923a..e0ab991bec 100644
&gt; --- a/toolchain/helpers.mk
&gt; +++ b/toolchain/helpers.mk
&gt; @@ -28,6 +28,10 @@ copy_toolchain_lib_root = \
&gt;&nbsp; 			elif test -f $${LIBPATH}; then \
&gt;&nbsp; 				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
&gt;&nbsp; 				break ; \
&gt; +			elif test -d $${LIBPATH}; then \
&gt; +				mkdir -p $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
&gt; +				cp -r $${LIBPATH}/* $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
&gt; +				break ; \

The proper fix would be to exclude directories in the find on line 13.

Indeed, LIBPATTERN is the argument to copy_toolchain_lib_root, and in
the case of external toolchains, is each item in TOOLCHAIN_EXTERNAL_LIBS,
the first of which is ld*.so.*, which we expect to only match files, but
in your case matches a directory.

As I understand it, ld.so.conf is not used at build time to find
libraries, so having it in the staging is useless.

Also note that having /etc/ld.so.conf.d in the target is not supported,
and that Buildroot will explicitly fail at the end of the build if it
sees such a directory, as it is not possible to have proper ldconfig
setup in cross-compilation; see the top-level Makefile, lines 753-756,
and commit 9c4072348960 (Makefile: drop ldconfig handling). And so it is
even weirder that your change makes the build succeed...

Regards,
Yann E. MORIN.

&gt;&nbsp; 			else \
&gt;&nbsp; 				exit -1; \
&gt;&nbsp; 			fi; \
&gt; -- 
&gt; 2.40.0
&gt; 
&gt; _______________________________________________
&gt; buildroot mailing list
&gt; buildroot@buildroot.org
&gt; https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|&nbsp; Yann E. MORIN&nbsp; | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software&nbsp; Designer | \ / CAMPAIGN&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; ___&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| +33 561 099 427 `------------.-------:&nbsp; X&nbsp; AGAINST&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; \e/&nbsp; There is no&nbsp; |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; v&nbsp;&nbsp; conspiracy.&nbsp; |
'------------------------------^-------^------------------^--------------------'

[-- Attachment #1.2: Type: text/html, Size: 6796 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] 回复： [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory

[Yann E. MORIN]

JiangXin, All,

On 2023-04-24 15:56 +0800, JiangXin spake thusly:
> I build rootfs in docker container based on ubuntu22.04, host arch is x86_64.
> I chose the gcc downloaded by apt to build. 

Ah, we do not support distro toolchains, see the manual, chapter 6.1.2.
External toolchain backend:

    https://buildroot.org/downloads/manual/manual.html#external-toolchain-backend

    We also do not support using the distribution toolchain (i.e. the
    gcc/binutils/C library installed by your distribution) as the
    toolchain to build software for the target. This is because your
    distribution toolchain is not a "pure" toolchain (i.e. only with
    the C/C++ library), so we cannot import it properly into the
    Buildroot build environment.

> After the shell command in helpers.mk is excuted, the following content is output:
> root@824b1b82dfae:~/buildroot-2022.02.11# find /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot -name
> "ld*.so.*"
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.cache
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/ld-linux-x86-64.so.2
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/lib/ld-linux-x86-64.so.2
> root@824b1b82dfae:~/buildroot-2022.02.11# ls -l
> /root/buildroot-2022.02.11/output/host/x86_64-buildroot-linux-gnu/sysroot/etc/ld.so.conf.d
> total 12
> -rw-r--r-- 1 root root  38 Mar  5  2022 fakeroot-x86_64-linux-gnu.conf
> -rw-r--r-- 1 root root  44 Sep 24  2021 libc.conf
> -rw-r--r-- 1 root root 100 Sep 28  2021 x86_64-linux-gnu.conf
> I added the echo on lines 753-756 of the top-level Makefile, and the build 
> log shows it doesn't get there.

This is weird, because that part should be unconditional. So, it looks
like the build may in fact not be finished successfully. Could you share
the last 50-or so last lines of the log?

> Should the processing of directories be added
> in toolchain/helpers.mk?

As I explained previously, and as confirmed above, support for
ld.so.conf should not be added, and thus the real solution is to
exclude directories in toolchain/helpers.mk line 13:

    find $(STAGING_DIR)/ -name "$${LIBPATTERN}" -not -type d 2>/dev/null

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] 回复： 回复： [PATCH 1/1] toolchain/helpers.mk: add the processing when LIBPATH is a directory

[Arnout Vandecappelle via buildroot]

  Hi JiangXin,

  Please don't top-post, but reply inline like I do below. Note that this 
requires configuring your mailer to quote replies properly.


On 25/04/2023 04:25, JiangXin wrote:
[snip]
> Is it possible for buildroot to support distro toolchains? I'm interested in 
> using my
> spare time to do that. It can save a little time for low performance processors.

  You can use one of the Bootlin toolchains. It's the default if you select to 
use an external toolchain. I can't imagine that you have a configuration for 
which a distro toolchain would exist, but no Bootlin toolchain.

  Regards,
  Arnout

[snip]
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot