Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/2] gitlab-ci: fix pipelines with the newer docker image (branch yem/pipelines)
@ 2023-02-07 21:33 Yann E. MORIN
  2023-02-07 21:33 ` [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates Yann E. MORIN
  2023-02-07 21:33 ` [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines Yann E. MORIN
  0 siblings, 2 replies; 7+ messages in thread
From: Yann E. MORIN @ 2023-02-07 21:33 UTC (permalink / raw)
  To: buildroot; +Cc: Romain Naour, Yann E . MORIN, Thomas Petazzoni

Hello All!

TL;DR: the newer docker image we use in our pipelines uses a newer git
version that is more strict with the ownership of the files in the git
repository checkout, which makes our check jobs fail.

Long story: see the individual commits. ;-)

Regards,
Yann E. MORIN.


----------------------------------------------------------------
Yann E. MORIN (2):
      gitlab-ci: don't use before_script in job templates
      gitlab-ci: fix pipelines

 support/misc/gitlab-ci.yml.in | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates
  2023-02-07 21:33 [Buildroot] [PATCH 0/2] gitlab-ci: fix pipelines with the newer docker image (branch yem/pipelines) Yann E. MORIN
@ 2023-02-07 21:33 ` Yann E. MORIN
  2023-02-07 21:47   ` Romain Naour
  2023-02-07 21:33 ` [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines Yann E. MORIN
  1 sibling, 1 reply; 7+ messages in thread
From: Yann E. MORIN @ 2023-02-07 21:33 UTC (permalink / raw)
  To: buildroot; +Cc: Romain Naour, Yann E. MORIN, Thomas Petazzoni

When gitlab prepares a job to run, it checks out the repository with a
non-root user, and spawns a container that runs as root, with some UID
mapping that makes the files be owned by root in the container. However,
our pipelines run as a nont-root user.

Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
the docker image that is used to run in our pipelines.

That new image includes a git version that is stricter about the
ownership of the git tree it is acting in: git aborts in error when the
user running it does not own the repository.

We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
rules, so they all fail (in various ways).

To fix this, we either need to fix the ownership or tell git to ignore
the situation. In either case, we'll need to run a scriptlet before all
our jobs.

Gitlab-ci allows to provide a global before_script, that is inherited by
all jobs. However, some of our jobs already declare a before_script, and
that would shadow the global before_Scrikpt.

There is no technical reason to do our before_script separately from
the actual script, so we move the code from the before_scripts to the
corresponding scripts.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/misc/gitlab-ci.yml.in | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
index 0ccf36665e..9c1faf0d5f 100644
--- a/support/misc/gitlab-ci.yml.in
+++ b/support/misc/gitlab-ci.yml.in
@@ -23,9 +23,8 @@
         - utils/check-symbols
 
 .defconfig_check:
-    before_script:
+    script:
         - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
-    script:
         - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
         - make ${DEFCONFIG_NAME}
         - support/scripts/check-dotconfig.py .config configs/${DEFCONFIG_NAME}
@@ -44,10 +43,9 @@
       }
 
 .defconfig_base:
-    before_script:
+    script:
         - DEFCONFIG_NAME=${CI_JOB_NAME}
         - OUTPUT_DIR=output
-    script:
         - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
         - make ${DEFCONFIG_NAME}
         - ./support/scripts/check-dotconfig.py .config ./configs/${DEFCONFIG_NAME}
@@ -72,13 +70,12 @@
             - runtime-test.log
 
 .runtime_test_base:
-    before_script:
-        - TEST_CASE_NAME=${CI_JOB_NAME}
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
     # Multiply every emulator timeout by 10 to avoid sporadic failures in
     # elastic runners.
     script:
+        - TEST_CASE_NAME=${CI_JOB_NAME}
         - echo "Starting runtime test ${TEST_CASE_NAME}"
         - ./support/testing/run-tests -o test-output/ -d test-dl/ -k --timeout-multiplier 10 ${TEST_CASE_NAME}
     artifacts:
@@ -91,9 +88,8 @@
 
 .test_pkg:
     stage: build
-    before_script:
+    script:
         - OUTPUT_DIR=${CI_JOB_NAME}
-    script:
         - echo "Configure Buildroot for ${OUTPUT_DIR}"
         - make O=${OUTPUT_DIR} syncconfig
         - make O=${OUTPUT_DIR} savedefconfig
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines
  2023-02-07 21:33 [Buildroot] [PATCH 0/2] gitlab-ci: fix pipelines with the newer docker image (branch yem/pipelines) Yann E. MORIN
  2023-02-07 21:33 ` [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates Yann E. MORIN
@ 2023-02-07 21:33 ` Yann E. MORIN
  2023-02-07 21:57   ` Romain Naour
  1 sibling, 1 reply; 7+ messages in thread
From: Yann E. MORIN @ 2023-02-07 21:33 UTC (permalink / raw)
  To: buildroot; +Cc: Romain Naour, Yann E. MORIN, Thomas Petazzoni

When gitlab prepares a job to run, it checks out the repository with a
non-root user, and spawns a container that runs as root, with some UID
mapping that makes the files be owned by root in the container. However,
our pipelines run as a nont-root user.

Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
the docker image that is used to run in our pipelines.

That new image includes a git version that is stricter about the
ownership of the git tree it is acting in: git aborts in error when the
user running it does not own the repository.

We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
rules, so they all fail (in various ways).

To fix this, we either need to fix the ownership or tell git to ignore
the situation.

It is most probably impossible to change the ownership of the files: we
run as non-root,and the files belong to root (in the container). So
we're stuck.

The alternative, is to do as git suggest, and tell it to ignore the
situation. In a local setup, this woujld be very insecure, but in the
pipelines, this is in a throw-away container, wehre a single user exists
and is running, so we don't care much (if at all).

Add a global before_script that registers the git config to ignore
ownership issues in the buildroot repository; see [0] for the definition
of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
with CI_BUILDS_DIR (the documentation is unclear about that point).

[0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
[1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/misc/gitlab-ci.yml.in | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
index 9c1faf0d5f..38aca31fb5 100644
--- a/support/misc/gitlab-ci.yml.in
+++ b/support/misc/gitlab-ci.yml.in
@@ -1,3 +1,6 @@
+before_script:
+  - git config --global --add safe.directory ${CI_PROJECT_DIR}
+
 .check-check-package_base:
     script:
         - python3 -m pytest -v utils/checkpackagelib/
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates
  2023-02-07 21:33 ` [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates Yann E. MORIN
@ 2023-02-07 21:47   ` Romain Naour
  2023-02-07 21:55     ` Yann E. MORIN
  0 siblings, 1 reply; 7+ messages in thread
From: Romain Naour @ 2023-02-07 21:47 UTC (permalink / raw)
  To: Yann E. MORIN, buildroot; +Cc: Romain Naour, Thomas Petazzoni

Hello Yann,

Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> When gitlab prepares a job to run, it checks out the repository with a
> non-root user, and spawns a container that runs as root, with some UID
> mapping that makes the files be owned by root in the container. However,
> our pipelines run as a nont-root user.
> 
> Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> the docker image that is used to run in our pipelines.
> 
> That new image includes a git version that is stricter about the
> ownership of the git tree it is acting in: git aborts in error when the
> user running it does not own the repository.
> 
> We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> rules, so they all fail (in various ways).

is flake8 has been updated too? flake8 may trigger new check-package warning now.
> 
> To fix this, we either need to fix the ownership or tell git to ignore
> the situation. In either case, we'll need to run a scriptlet before all
> our jobs.
> 
> Gitlab-ci allows to provide a global before_script, that is inherited by
> all jobs. However, some of our jobs already declare a before_script, and
> that would shadow the global before_Scrikpt.

before_Scrikpt/before_scripts

> 
> There is no technical reason to do our before_script separately from
> the actual script, so we move the code from the before_scripts to the
> corresponding scripts.
> 

Reviewed-by: Romain Naour <romain.naour@smile.fr>

Best regards,
Romain


> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Romain Naour <romain.naour@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/misc/gitlab-ci.yml.in | 12 ++++--------
>  1 file changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> index 0ccf36665e..9c1faf0d5f 100644
> --- a/support/misc/gitlab-ci.yml.in
> +++ b/support/misc/gitlab-ci.yml.in
> @@ -23,9 +23,8 @@
>          - utils/check-symbols
>  
>  .defconfig_check:
> -    before_script:
> +    script:
>          - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
> -    script:
>          - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
>          - make ${DEFCONFIG_NAME}
>          - support/scripts/check-dotconfig.py .config configs/${DEFCONFIG_NAME}
> @@ -44,10 +43,9 @@
>        }
>  
>  .defconfig_base:
> -    before_script:
> +    script:
>          - DEFCONFIG_NAME=${CI_JOB_NAME}
>          - OUTPUT_DIR=output
> -    script:
>          - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
>          - make ${DEFCONFIG_NAME}
>          - ./support/scripts/check-dotconfig.py .config ./configs/${DEFCONFIG_NAME}
> @@ -72,13 +70,12 @@
>              - runtime-test.log
>  
>  .runtime_test_base:
> -    before_script:
> -        - TEST_CASE_NAME=${CI_JOB_NAME}
>      # Keep build directories so the rootfs can be an artifact of the job. The
>      # runner will clean up those files for us.
>      # Multiply every emulator timeout by 10 to avoid sporadic failures in
>      # elastic runners.
>      script:
> +        - TEST_CASE_NAME=${CI_JOB_NAME}
>          - echo "Starting runtime test ${TEST_CASE_NAME}"
>          - ./support/testing/run-tests -o test-output/ -d test-dl/ -k --timeout-multiplier 10 ${TEST_CASE_NAME}
>      artifacts:
> @@ -91,9 +88,8 @@
>  
>  .test_pkg:
>      stage: build
> -    before_script:
> +    script:
>          - OUTPUT_DIR=${CI_JOB_NAME}
> -    script:
>          - echo "Configure Buildroot for ${OUTPUT_DIR}"
>          - make O=${OUTPUT_DIR} syncconfig
>          - make O=${OUTPUT_DIR} savedefconfig

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates
  2023-02-07 21:47   ` Romain Naour
@ 2023-02-07 21:55     ` Yann E. MORIN
  0 siblings, 0 replies; 7+ messages in thread
From: Yann E. MORIN @ 2023-02-07 21:55 UTC (permalink / raw)
  To: Romain Naour; +Cc: Romain Naour, Thomas Petazzoni, buildroot

Romain, All,

On 2023-02-07 22:47 +0100, Romain Naour spake thusly:
> Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> > When gitlab prepares a job to run, it checks out the repository with a
> > non-root user, and spawns a container that runs as root, with some UID
> > mapping that makes the files be owned by root in the container. However,
> > our pipelines run as a nont-root user.
> > 
> > Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> > the docker image that is used to run in our pipelines.
> > 
> > That new image includes a git version that is stricter about the
> > ownership of the git tree it is acting in: git aborts in error when the
> > user running it does not own the repository.
> > 
> > We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> > rules, so they all fail (in various ways).
> 
> is flake8 has been updated too? flake8 may trigger new check-package warning now.

Yes, there is a newer flake8 too, but there is no new error detected:

    https://gitlab.com/ymorin/buildroot/-/pipelines/770406457

> > 
> > To fix this, we either need to fix the ownership or tell git to ignore
> > the situation. In either case, we'll need to run a scriptlet before all
> > our jobs.
> > 
> > Gitlab-ci allows to provide a global before_script, that is inherited by
> > all jobs. However, some of our jobs already declare a before_script, and
> > that would shadow the global before_Scrikpt.
> before_Scrikpt/before_scripts

Thanks Thomas said he'd fix that when applying. ;-]

> > There is no technical reason to do our before_script separately from
> > the actual script, so we move the code from the before_scripts to the
> > corresponding scripts.
> Reviewed-by: Romain Naour <romain.naour@smile.fr>

Thanks!

Regards,
Yann E. MORIN.

> Best regards,
> Romain
> 
> 
> > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> > Cc: Romain Naour <romain.naour@gmail.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > ---
> >  support/misc/gitlab-ci.yml.in | 12 ++++--------
> >  1 file changed, 4 insertions(+), 8 deletions(-)
> > 
> > diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> > index 0ccf36665e..9c1faf0d5f 100644
> > --- a/support/misc/gitlab-ci.yml.in
> > +++ b/support/misc/gitlab-ci.yml.in
> > @@ -23,9 +23,8 @@
> >          - utils/check-symbols
> >  
> >  .defconfig_check:
> > -    before_script:
> > +    script:
> >          - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
> > -    script:
> >          - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
> >          - make ${DEFCONFIG_NAME}
> >          - support/scripts/check-dotconfig.py .config configs/${DEFCONFIG_NAME}
> > @@ -44,10 +43,9 @@
> >        }
> >  
> >  .defconfig_base:
> > -    before_script:
> > +    script:
> >          - DEFCONFIG_NAME=${CI_JOB_NAME}
> >          - OUTPUT_DIR=output
> > -    script:
> >          - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
> >          - make ${DEFCONFIG_NAME}
> >          - ./support/scripts/check-dotconfig.py .config ./configs/${DEFCONFIG_NAME}
> > @@ -72,13 +70,12 @@
> >              - runtime-test.log
> >  
> >  .runtime_test_base:
> > -    before_script:
> > -        - TEST_CASE_NAME=${CI_JOB_NAME}
> >      # Keep build directories so the rootfs can be an artifact of the job. The
> >      # runner will clean up those files for us.
> >      # Multiply every emulator timeout by 10 to avoid sporadic failures in
> >      # elastic runners.
> >      script:
> > +        - TEST_CASE_NAME=${CI_JOB_NAME}
> >          - echo "Starting runtime test ${TEST_CASE_NAME}"
> >          - ./support/testing/run-tests -o test-output/ -d test-dl/ -k --timeout-multiplier 10 ${TEST_CASE_NAME}
> >      artifacts:
> > @@ -91,9 +88,8 @@
> >  
> >  .test_pkg:
> >      stage: build
> > -    before_script:
> > +    script:
> >          - OUTPUT_DIR=${CI_JOB_NAME}
> > -    script:
> >          - echo "Configure Buildroot for ${OUTPUT_DIR}"
> >          - make O=${OUTPUT_DIR} syncconfig
> >          - make O=${OUTPUT_DIR} savedefconfig
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines
  2023-02-07 21:33 ` [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines Yann E. MORIN
@ 2023-02-07 21:57   ` Romain Naour
  2023-02-07 22:15     ` Yann E. MORIN
  0 siblings, 1 reply; 7+ messages in thread
From: Romain Naour @ 2023-02-07 21:57 UTC (permalink / raw)
  To: Yann E. MORIN, buildroot; +Cc: Romain Naour, Thomas Petazzoni

Hello Yann,

Is "fix pipelines" your WIP patch subject ?
"gitlab-ci: ignore ownership of the git tree" seems better?

Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> When gitlab prepares a job to run, it checks out the repository with a
> non-root user, and spawns a container that runs as root, with some UID
> mapping that makes the files be owned by root in the container. However,
> our pipelines run as a nont-root user.
> 
> Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> the docker image that is used to run in our pipelines.
> 
> That new image includes a git version that is stricter about the
> ownership of the git tree it is acting in: git aborts in error when the
> user running it does not own the repository.
> 
> We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> rules, so they all fail (in various ways).
> 
> To fix this, we either need to fix the ownership or tell git to ignore
> the situation.
> 
> It is most probably impossible to change the ownership of the files: we
> run as non-root,and the files belong to root (in the container). So
> we're stuck.
> 
> The alternative, is to do as git suggest, and tell it to ignore the
> situation. In a local setup, this woujld be very insecure, but in the
> pipelines, this is in a throw-away container, wehre a single user exists
> and is running, so we don't care much (if at all).
> 
> Add a global before_script that registers the git config to ignore
> ownership issues in the buildroot repository; see [0] for the definition
> of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
> in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
> with CI_BUILDS_DIR (the documentation is unclear about that point).
> 
> [0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
> [1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section

Reviewed-by: Romain Naour <romain.naour@smile.fr>


> 
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Romain Naour <romain.naour@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/misc/gitlab-ci.yml.in | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> index 9c1faf0d5f..38aca31fb5 100644
> --- a/support/misc/gitlab-ci.yml.in
> +++ b/support/misc/gitlab-ci.yml.in
> @@ -1,3 +1,6 @@
> +before_script:
> +  - git config --global --add safe.directory ${CI_PROJECT_DIR}

Note: I checked if we have the issue using the utils/docker-run directly,
enabling safe.directory is not necessary.

Best regards,
Romain


> +
>  .check-check-package_base:
>      script:
>          - python3 -m pytest -v utils/checkpackagelib/

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines
  2023-02-07 21:57   ` Romain Naour
@ 2023-02-07 22:15     ` Yann E. MORIN
  0 siblings, 0 replies; 7+ messages in thread
From: Yann E. MORIN @ 2023-02-07 22:15 UTC (permalink / raw)
  To: Romain Naour; +Cc: Romain Naour, Thomas Petazzoni, buildroot

Romain, All,

On 2023-02-07 22:57 +0100, Romain Naour spake thusly:
> Is "fix pipelines" your WIP patch subject ?
> "gitlab-ci: ignore ownership of the git tree" seems better?

No, I really intended to write "fix pipeline".

Indeed, the subject of a commit log should be a summary of the semantic
change, not of how it is done.

Thanks for the review! :-)

Regards,
Yann E. MORIN.

> Le 07/02/2023 à 22:33, Yann E. MORIN a écrit :
> > When gitlab prepares a job to run, it checks out the repository with a
> > non-root user, and spawns a container that runs as root, with some UID
> > mapping that makes the files be owned by root in the container. However,
> > our pipelines run as a nont-root user.
> > 
> > Commit bde165f7ad (.gitlab-ci.yml: update Docker image to use) updated
> > the docker image that is used to run in our pipelines.
> > 
> > That new image includes a git version that is stricter about the
> > ownership of the git tree it is acting in: git aborts in error when the
> > user running it does not own the repository.
> > 
> > We use `git ls-tree` quite a lot in our check-{flake8,package,symbols}
> > rules, so they all fail (in various ways).
> > 
> > To fix this, we either need to fix the ownership or tell git to ignore
> > the situation.
> > 
> > It is most probably impossible to change the ownership of the files: we
> > run as non-root,and the files belong to root (in the container). So
> > we're stuck.
> > 
> > The alternative, is to do as git suggest, and tell it to ignore the
> > situation. In a local setup, this woujld be very insecure, but in the
> > pipelines, this is in a throw-away container, wehre a single user exists
> > and is running, so we don't care much (if at all).
> > 
> > Add a global before_script that registers the git config to ignore
> > ownership issues in the buildroot repository; see [0] for the definition
> > of the CI_PROJECT_DIR variable. Note: unlike what is said in there, and
> > in [1], the value actually seen in CI_PROJECT_DIR is already prefixed
> > with CI_BUILDS_DIR (the documentation is unclear about that point).
> > 
> > [0] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
> > [1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
> 
> Reviewed-by: Romain Naour <romain.naour@smile.fr>
> 
> 
> > 
> > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> > Cc: Romain Naour <romain.naour@gmail.com>
> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > ---
> >  support/misc/gitlab-ci.yml.in | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/support/misc/gitlab-ci.yml.in b/support/misc/gitlab-ci.yml.in
> > index 9c1faf0d5f..38aca31fb5 100644
> > --- a/support/misc/gitlab-ci.yml.in
> > +++ b/support/misc/gitlab-ci.yml.in
> > @@ -1,3 +1,6 @@
> > +before_script:
> > +  - git config --global --add safe.directory ${CI_PROJECT_DIR}
> 
> Note: I checked if we have the issue using the utils/docker-run directly,
> enabling safe.directory is not necessary.
> 
> Best regards,
> Romain
> 
> 
> > +
> >  .check-check-package_base:
> >      script:
> >          - python3 -m pytest -v utils/checkpackagelib/
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2023-02-07 22:16 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-07 21:33 [Buildroot] [PATCH 0/2] gitlab-ci: fix pipelines with the newer docker image (branch yem/pipelines) Yann E. MORIN
2023-02-07 21:33 ` [Buildroot] [PATCH 1/2] gitlab-ci: don't use before_script in job templates Yann E. MORIN
2023-02-07 21:47   ` Romain Naour
2023-02-07 21:55     ` Yann E. MORIN
2023-02-07 21:33 ` [Buildroot] [PATCH 2/2] gitlab-ci: fix pipelines Yann E. MORIN
2023-02-07 21:57   ` Romain Naour
2023-02-07 22:15     ` Yann E. MORIN

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox