[Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Peter Korsgaard]

A number of packages try to detect if they are running in a git repo and run
git describe at build time instead of using the hard coded version number if
it succeed, leading to odd version numbers as they end up picking up the
Buildroot git version if building inside a Buildroot git checkout, E.G.:

rauc --version
rauc 2023.11-562-g9c954953b4+

This is because rauc builds with meson and uses vcs_tag:

https://github.com/rauc/rauc/blob/v1.11/meson.build#L168-L171

https://mesonbuild.com/Reference-manual_functions.html#vcs_tag

Another example is micropython, where we already work around it by passing
GIT_DIR=.

In the context of Buildroot the packages are never built in their own git
checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
directory tree and finds the Buildroot git repo, which fixes the rauc (and
similar) issues.

>>> rauc 1.11 Building
..
ninja: Entering directory `/home/peko/source/buildroot/output-rauc/build/rauc-1.11//build'
[1/29] Generating version.h with a custom command
fatal: not a git repository: '.'

cat output-rauc/build/rauc-1.11/build/version.h
 #define PACKAGE_STRING "rauc 1.11"

 #define PACKAGE_VERSION "1.11"

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
 package/Makefile.in | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/package/Makefile.in b/package/Makefile.in
index 9fbe960759..3e276d23d6 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -263,7 +263,9 @@ export PERL=$(shell which perl)
 # finds this perl module by exporting the proper value for PERL5LIB.
 export PERL5LIB=$(HOST_DIR)/lib/perl
 
-TARGET_MAKE_ENV = PATH=$(BR_PATH)
+TARGET_MAKE_ENV = \
+	GIT_DIR=. \
+	PATH=$(BR_PATH)
 
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
@@ -307,6 +309,7 @@ TARGET_CONFIGURE_OPTS = \
 
 
 HOST_MAKE_ENV = \
+	GIT_DIR=. \
 	PATH=$(BR_PATH) \
 	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
 	PKG_CONFIG_SYSROOT_DIR="/" \
-- 
2.39.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] package/micropython: drop GIT_DIR=. workaround

[Peter Korsgaard]

Now that it is handled globally in TARGET_MAKE_ENV.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
---
 package/micropython/micropython.mk | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/package/micropython/micropython.mk b/package/micropython/micropython.mk
index e6eb536ea5..3d356c51d8 100644
--- a/package/micropython/micropython.mk
+++ b/package/micropython/micropython.mk
@@ -14,11 +14,6 @@ MICROPYTHON_LICENSE = MIT, BSD-1-Clause, BSD-3-Clause, Zlib
 MICROPYTHON_LICENSE_FILES = LICENSE
 MICROPYTHON_DEPENDENCIES = host-pkgconf libffi host-python3
 
-# Set GIT_DIR so package won't use buildroot's version number
-MICROPYTHON_MAKE_ENV = \
-	$(TARGET_MAKE_ENV) \
-	GIT_DIR=.
-
 # Use fallback implementation for exception handling on architectures that don't
 # have explicit support.
 ifeq ($(BR2_i386)$(BR2_x86_64)$(BR2_arm)$(BR2_armeb),)
-- 
2.39.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/micropython: drop GIT_DIR=. workaround

[Yann E. MORIN]

Peter, All,

On 2024-01-11 11:56 +0100, Peter Korsgaard spake thusly:
> Now that it is handled globally in TARGET_MAKE_ENV.
> 
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  package/micropython/micropython.mk | 5 -----
>  1 file changed, 5 deletions(-)
> 
> diff --git a/package/micropython/micropython.mk b/package/micropython/micropython.mk
> index e6eb536ea5..3d356c51d8 100644
> --- a/package/micropython/micropython.mk
> +++ b/package/micropython/micropython.mk
> @@ -14,11 +14,6 @@ MICROPYTHON_LICENSE = MIT, BSD-1-Clause, BSD-3-Clause, Zlib
>  MICROPYTHON_LICENSE_FILES = LICENSE
>  MICROPYTHON_DEPENDENCIES = host-pkgconf libffi host-python3
>  
> -# Set GIT_DIR so package won't use buildroot's version number
> -MICROPYTHON_MAKE_ENV = \
> -	$(TARGET_MAKE_ENV) \
> -	GIT_DIR=.
> -
>  # Use fallback implementation for exception handling on architectures that don't
>  # have explicit support.
>  ifeq ($(BR2_i386)$(BR2_x86_64)$(BR2_arm)$(BR2_armeb),)
> -- 
> 2.39.2
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/micropython: drop GIT_DIR=. workaround

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > Now that it is handled globally in TARGET_MAKE_ENV.
 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Committed to 2023.02.x and 2023.11.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/micropython: drop GIT_DIR=. workaround

[Yann E. MORIN]

Peter, All,

On 2024-01-11 11:56 +0100, Peter Korsgaard spake thusly:
> Now that it is handled globally in TARGET_MAKE_ENV.
> 
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
> ---
>  package/micropython/micropython.mk | 5 -----
>  1 file changed, 5 deletions(-)
> 
> diff --git a/package/micropython/micropython.mk b/package/micropython/micropython.mk
> index e6eb536ea5..3d356c51d8 100644
> --- a/package/micropython/micropython.mk
> +++ b/package/micropython/micropython.mk
> @@ -14,11 +14,6 @@ MICROPYTHON_LICENSE = MIT, BSD-1-Clause, BSD-3-Clause, Zlib
>  MICROPYTHON_LICENSE_FILES = LICENSE
>  MICROPYTHON_DEPENDENCIES = host-pkgconf libffi host-python3
>  
> -# Set GIT_DIR so package won't use buildroot's version number
> -MICROPYTHON_MAKE_ENV = \
> -	$(TARGET_MAKE_ENV) \
> -	GIT_DIR=.

I was way too sloppy when applying this patch: MICROPYTHON_MAKE_ENV
previously contained $(TARGET_MAKE_ENV), but now it is totally unset,
and the expansions of it have not been replaced with TARGET_MAKE_ENV,
leading to all sorts of build faiulures, like:

    LINK build-standard/micropython
    arm-linux-gcc: ERROR: unsafe header/library path used in cross-compilation: '-L/usr/lib64/../lib64'

I suspect it is also the issue reported by Julien in his micropython
runtime test:

    https://patchwork.ozlabs.org/project/buildroot/patch/20240116210556.589407-1-ju.o@free.fr/

I'll push a fixup patch that uses TARGET_MAKE_ENV

Regards,
Yann E. MORIN.

>  # Use fallback implementation for exception handling on architectures that don't
>  # have explicit support.
>  ifeq ($(BR2_i386)$(BR2_x86_64)$(BR2_arm)$(BR2_armeb),)
> -- 
> 2.39.2
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] package/micropython: drop GIT_DIR=. workaround

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > Peter, All,
 > On 2024-01-11 11:56 +0100, Peter Korsgaard spake thusly:
 >> Now that it is handled globally in TARGET_MAKE_ENV.
 >> 
 >> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
 >> ---
 >> package/micropython/micropython.mk | 5 -----
 >> 1 file changed, 5 deletions(-)
 >> 
 >> diff --git a/package/micropython/micropython.mk b/package/micropython/micropython.mk
 >> index e6eb536ea5..3d356c51d8 100644
 >> --- a/package/micropython/micropython.mk
 >> +++ b/package/micropython/micropython.mk
 >> @@ -14,11 +14,6 @@ MICROPYTHON_LICENSE = MIT, BSD-1-Clause, BSD-3-Clause, Zlib
 >> MICROPYTHON_LICENSE_FILES = LICENSE
 >> MICROPYTHON_DEPENDENCIES = host-pkgconf libffi host-python3
 >> 
 >> -# Set GIT_DIR so package won't use buildroot's version number
 >> -MICROPYTHON_MAKE_ENV = \
 >> -	$(TARGET_MAKE_ENV) \
 >> -	GIT_DIR=.

 > I was way too sloppy when applying this patch: MICROPYTHON_MAKE_ENV
 > previously contained $(TARGET_MAKE_ENV), but now it is totally unset,
 > and the expansions of it have not been replaced with TARGET_MAKE_ENV,
 > leading to all sorts of build faiulures, like:

 >     LINK build-standard/micropython
 >     arm-linux-gcc: ERROR: unsafe header/library path used in
 > cross-compilation: '-L/usr/lib64/../lib64'

 > I suspect it is also the issue reported by Julien in his micropython
 > runtime test:

 >     https://patchwork.ozlabs.org/project/buildroot/patch/20240116210556.589407-1-ju.o@free.fr/

 > I'll push a fixup patch that uses TARGET_MAKE_ENV

Argh, thanks! Sorry.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Yann E. MORIN]

Peter, All,

On 2024-01-11 11:56 +0100, Peter Korsgaard spake thusly:
> A number of packages try to detect if they are running in a git repo and run
> git describe at build time instead of using the hard coded version number if
> it succeed, leading to odd version numbers as they end up picking up the
> Buildroot git version if building inside a Buildroot git checkout, E.G.:
> 
> rauc --version
> rauc 2023.11-562-g9c954953b4+
> 
> This is because rauc builds with meson and uses vcs_tag:
> 
> https://github.com/rauc/rauc/blob/v1.11/meson.build#L168-L171
> 
> https://mesonbuild.com/Reference-manual_functions.html#vcs_tag
> 
> Another example is micropython, where we already work around it by passing
> GIT_DIR=.
> 
> In the context of Buildroot the packages are never built in their own git
> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
> directory tree and finds the Buildroot git repo, which fixes the rauc (and
> similar) issues.
> 
> >>> rauc 1.11 Building
> ..
> ninja: Entering directory `/home/peko/source/buildroot/output-rauc/build/rauc-1.11//build'
> [1/29] Generating version.h with a custom command
> fatal: not a git repository: '.'
> 
> cat output-rauc/build/rauc-1.11/build/version.h
>  #define PACKAGE_STRING "rauc 1.11"
> 
>  #define PACKAGE_VERSION "1.11"
> 
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Applied to master, thanks.

Regards,
Yann E. MORIN.

> ---
>  package/Makefile.in | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 9fbe960759..3e276d23d6 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -263,7 +263,9 @@ export PERL=$(shell which perl)
>  # finds this perl module by exporting the proper value for PERL5LIB.
>  export PERL5LIB=$(HOST_DIR)/lib/perl
>  
> -TARGET_MAKE_ENV = PATH=$(BR_PATH)
> +TARGET_MAKE_ENV = \
> +	GIT_DIR=. \
> +	PATH=$(BR_PATH)
>  
>  TARGET_CONFIGURE_OPTS = \
>  	$(TARGET_MAKE_ENV) \
> @@ -307,6 +309,7 @@ TARGET_CONFIGURE_OPTS = \
>  
>  
>  HOST_MAKE_ENV = \
> +	GIT_DIR=. \
>  	PATH=$(BR_PATH) \
>  	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
>  	PKG_CONFIG_SYSROOT_DIR="/" \
> -- 
> 2.39.2
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Arnout Vandecappelle via buildroot]

On 11/01/2024 11:56, Peter Korsgaard wrote:
> A number of packages try to detect if they are running in a git repo and run
> git describe at build time instead of using the hard coded version number if
> it succeed, leading to odd version numbers as they end up picking up the
> Buildroot git version if building inside a Buildroot git checkout, E.G.:
> 
> rauc --version
> rauc 2023.11-562-g9c954953b4+
> 
> This is because rauc builds with meson and uses vcs_tag:
> 
> https://github.com/rauc/rauc/blob/v1.11/meson.build#L168-L171
> 
> https://mesonbuild.com/Reference-manual_functions.html#vcs_tag
> 
> Another example is micropython, where we already work around it by passing
> GIT_DIR=.
> 
> In the context of Buildroot the packages are never built in their own git
> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
> directory tree and finds the Buildroot git repo, which fixes the rauc (and
> similar) issues.
> 
>>>> rauc 1.11 Building
> ..
> ninja: Entering directory `/home/peko/source/buildroot/output-rauc/build/rauc-1.11//build'
> [1/29] Generating version.h with a custom command
> fatal: not a git repository: '.'
> 
> cat output-rauc/build/rauc-1.11/build/version.h
>   #define PACKAGE_STRING "rauc 1.11"
> 
>   #define PACKAGE_VERSION "1.11"
> 
> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
> ---
>   package/Makefile.in | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 9fbe960759..3e276d23d6 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -263,7 +263,9 @@ export PERL=$(shell which perl)
>   # finds this perl module by exporting the proper value for PERL5LIB.
>   export PERL5LIB=$(HOST_DIR)/lib/perl
>   
> -TARGET_MAKE_ENV = PATH=$(BR_PATH)
> +TARGET_MAKE_ENV = \
> +	GIT_DIR=. \
> +	PATH=$(BR_PATH)

  There are a number of infras that don't use TARGET_MAKE_ENV: golang, luarocks, 
perl, and python. Of these, I think only golang has a big risk of having the 
same issue. So shouldn't we add it there as well? And perhaps to python as well.


  Regards,
  Arnout

>   
>   TARGET_CONFIGURE_OPTS = \
>   	$(TARGET_MAKE_ENV) \
> @@ -307,6 +309,7 @@ TARGET_CONFIGURE_OPTS = \
>   
>   
>   HOST_MAKE_ENV = \
> +	GIT_DIR=. \
>   	PATH=$(BR_PATH) \
>   	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
>   	PKG_CONFIG_SYSROOT_DIR="/" \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> # finds this perl module by exporting the proper value for PERL5LIB.
 >> export PERL5LIB=$(HOST_DIR)/lib/perl
 >> -TARGET_MAKE_ENV = PATH=$(BR_PATH)
 >> +TARGET_MAKE_ENV = \
 >> +	GIT_DIR=. \
 >> +	PATH=$(BR_PATH)

 >  There are a number of infras that don't use TARGET_MAKE_ENV: golang,
 >  luarocks, perl, and python. Of these, I think only golang has a big
 >  risk of having the same issue. So shouldn't we add it there as well?
 >  And perhaps to python as well.

Possibly, yes. We have to be a bit careful to not cause issues with the
module vendoring, but it should be doable.

Will you send patches?

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Peter Korsgaard]

>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

 > A number of packages try to detect if they are running in a git repo and run
 > git describe at build time instead of using the hard coded version number if
 > it succeed, leading to odd version numbers as they end up picking up the
 > Buildroot git version if building inside a Buildroot git checkout, E.G.:

 > rauc --version
 > rauc 2023.11-562-g9c954953b4+

 > This is because rauc builds with meson and uses vcs_tag:

 > https://github.com/rauc/rauc/blob/v1.11/meson.build#L168-L171

 > https://mesonbuild.com/Reference-manual_functions.html#vcs_tag

 > Another example is micropython, where we already work around it by passing
 > GIT_DIR=.

 > In the context of Buildroot the packages are never built in their own git
 > checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
 > directory tree and finds the Buildroot git repo, which fixes the rauc (and
 > similar) issues.

 >>>> rauc 1.11 Building
 > ..
 > ninja: Entering directory `/home/peko/source/buildroot/output-rauc/build/rauc-1.11//build'
 > [1/29] Generating version.h with a custom command
 > fatal: not a git repository: '.'

 > cat output-rauc/build/rauc-1.11/build/version.h
 >  #define PACKAGE_STRING "rauc 1.11"

 >  #define PACKAGE_VERSION "1.11"

 > Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

Committed to 2023.02.x and 2023.11.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Jan Kundrát via buildroot]

> In the context of Buildroot the packages are never built in their own git
> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
> directory tree and finds the Buildroot git repo, which fixes the rauc (and
> similar) issues.

This has caused a regression for us.

We're using Zuul CI [1] which is set up so that any patch that's uploaded 
for review in any of our internal projects also triggers a test build 
through Buildroot. Some of these builds would use sequences of yet-unmerged 
patches, so tracking version info of each component is crucial, which is 
why we're puting all that into the target's /etc/os-release, as well as 
relaying on the build system to call an equivalent of a `git describe` for 
each of our projects.

For this feature, we're relying on a few hooks:

- Zuul itself produces the correct content as a git checkout in a scratch 
directory,

- Buildroot is configured via `local.mk` to use that checkout as a source 
[2]

- a custom hook sets up the redirection via the `gitdir` statement in the 
buidlroot's copy of the source directory [3], this redirection however  
takes effect at runtime (see [4])

- the build script of each app then simply runs `git describe` with a few 
flags for output formatting; the apps currently do not have a way of 
accepting some externally provided version strings.

Some of our projects use git submodules, which means that manipulating 
.gitdir has to be done carefuly. The TL;DR version is that this is a bit 
more complex than just putting a version override into a file. In short, 
there's already a git checkout that's been prepared with accurate version 
info, and it makes sense to use that info as-is -- or it has worked well 
for us in the past three years :), up until this commit.

How are you guys solving this problem in your CI systems? Is there a chance 
of undoing this global GIT_DIR which, unfortunately, takes precedence over 
the projects' own .gitdir mechanism?

With kind regards,
Jan

[1] https://zuul-ci.org/
[2] 
https://gerrit.cesnet.cz/plugins/gitiles/CzechLight/br2-external/+/b1066e39fe136c4996f7d13b6003a958c1ea40a2/dev-setup-git.sh#10
[3] 
https://gerrit.cesnet.cz/plugins/gitiles/CzechLight/br2-external/+/b1066e39fe136c4996f7d13b6003a958c1ea40a2/dev-setup-git.sh#31
[4] 
https://gerrit.cesnet.cz/plugins/gitiles/CzechLight/br2-external/+/7a9d8b0d8f3cefedf708368bba463c89556dfa9e%5E%21/
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Peter Korsgaard]

>>>>> "Jan" == Jan Kundrát via buildroot <buildroot@buildroot.org> writes:

 >> In the context of Buildroot the packages are never built in their own git
 >> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
 >> directory tree and finds the Buildroot git repo, which fixes the rauc (and
 >> similar) issues.

 > This has caused a regression for us.

Sorry to hear that.

 > We're using Zuul CI [1] which is set up so that any patch that's
 > uploaded for review in any of our internal projects also triggers a
 > test build through Buildroot. Some of these builds would use sequences
 > of yet-unmerged patches, so tracking version info of each component is
 > crucial, which is why we're puting all that into the target's
 > /etc/os-release, as well as relaying on the build system to call an
 > equivalent of a `git describe` for each of our projects.

 > For this feature, we're relying on a few hooks:

 > - Zuul itself produces the correct content as a git checkout in a
 >   scratch directory,

 > - Buildroot is configured via `local.mk` to use that checkout as a
 >   source [2]

 > - a custom hook sets up the redirection via the `gitdir` statement in
 >   the buidlroot's copy of the source directory [3], this redirection
 >   however  takes effect at runtime (see [4])

I don't understand why you do > \$(@D)/.git, I was would have
expected >> \$(@D)/.git/config but OK.


 > - the build script of each app then simply runs `git describe` with a
 >   few flags for output formatting; the apps currently do not have a
 >   way of accepting some externally provided version strings.

 > Some of our projects use git submodules, which means that manipulating
 > .gitdir has to be done carefuly. The TL;DR version is that this is a
 > bit more complex than just putting a version override into a file. In
 > short, there's already a git checkout that's been prepared with
 > accurate version info, and it makes sense to use that info as-is -- or
 > it has worked well for us in the past three years :), up until this
 > commit.

I guess those are local/custom packages that you have in your external
tree? Can you pass GIT_DIR=.git to them (or make them unset GIT_DIR) to
make them use your prepared .git directory?


 > How are you guys solving this problem in your CI systems?

This is not applicable to the CI done for Buildroot itself (where we
want to build the versions specified in the .mk files). For $WORK, I
have populated a file in a post-build script like your os-release file.


 > Is there a chance of undoing this global GIT_DIR which,
 > unfortunately, takes precedence over the projects' own .gitdir
 > mechanism?

The problem this change tried to fix is real, so the question is how we
can continue to fix this issue (without having to play whack-a-mole for
each package) which still supporting a use case like yours.

Thanks for the heads up!

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Jan Kundrát via buildroot]

> I guess those are local/custom packages that you have in your external
> tree? Can you pass GIT_DIR=.git to them (or make them unset GIT_DIR) to
> make them use your prepared .git directory?

Yes, these are local packages whose build recipes live in our br2-external 
tree.

I tried to unset the GIT_DIR from both $(PKG)_PRE_CONFIGURE_HOOKS and 
$(PKG)_PRE_BULD_HOOKS, but calling `unset GIT_DIR` has no effect in there. 
That makes sense, because it's used like this in the pkg-cmake.mk:

define $(2)_BUILD_CMDS
	$$(TARGET_MAKE_ENV) $$($$(PKG)_BUILD_ENV) $$(BR2_CMAKE) --build 
$$($$(PKG)_BUILDDIR) -j$(PARALLEL_JOBS) $$($$(PKG)_BUILD_OPTS)

...so it won't propagate because it's set explicitly just before the build 
command executes.

In the end, I did it like this:

  XXX_HACK_GIT_DIR = GIT_DIR=.git

...and then for each project:

  FOO_BUILD_ENV += $(XXX_HACK_GIT_DIR)
  BAR_BUILD_ENV += $(XXX_HACK_GIT_DIR)

Cheers,
Jan
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Mircea Gliga]

Hi all,

> In the context of Buildroot the packages are never built in their own git
> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
> directory tree and finds the Buildroot git repo, which fixes the rauc (and
> similar) issues.

>
> -TARGET_MAKE_ENV = PATH=$(BR_PATH)
> +TARGET_MAKE_ENV = \
> +       GIT_DIR=. \
> +       PATH=$(BR_PATH)
>
>  TARGET_CONFIGURE_OPTS = \
>         $(TARGET_MAKE_ENV) \
> @@ -307,6 +309,7 @@ TARGET_CONFIGURE_OPTS = \

This will also break the usage of package/environment-setup. The
GIT_DIR=. leaks in
the environment script `environment-setup` useful when a developer
wants to use a Buildroot
generated SDK to build an external project.
So, on a developer machine after sourcing the environment script, all
git commands fail:

$ git status
fatal: not a git repository: '.'

Best regards
Mircea
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] package/Makefile.in: set GIT_DIR=. in {HOST, TARGET}_MAKE_ENV

[Peter Korsgaard]

>>>>> "Mircea" == Mircea Gliga <gliga.mircea@gmail.com> writes:

 > Hi all,
 >> In the context of Buildroot the packages are never built in their own git
 >> checkout, so pass GIT_DIR=.  to ensure git doesn't walk back up the
 >> directory tree and finds the Buildroot git repo, which fixes the rauc (and
 >> similar) issues.

 >> 
 >> -TARGET_MAKE_ENV = PATH=$(BR_PATH)
 >> +TARGET_MAKE_ENV = \
 >> +       GIT_DIR=. \
 >> +       PATH=$(BR_PATH)
 >> 
 >> TARGET_CONFIGURE_OPTS = \
 >> $(TARGET_MAKE_ENV) \
 >> @@ -307,6 +309,7 @@ TARGET_CONFIGURE_OPTS = \

 > This will also break the usage of package/environment-setup. The
 > GIT_DIR=. leaks in
 > the environment script `environment-setup` useful when a developer
 > wants to use a Buildroot
 > generated SDK to build an external project.
 > So, on a developer machine after sourcing the environment script, all
 > git commands fail:

 > $ git status
 > fatal: not a git repository: '.'

Thanks for the report, I've sent a fix here:

https://patchwork.ozlabs.org/project/buildroot/patch/20240126093441.2130693-1-peter@korsgaard.com/

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot