* [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
@ 2023-10-02 14:54 Fabien Thomas
2023-10-02 17:16 ` [yocto] " Khem Raj
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Fabien Thomas @ 2023-10-02 14:54 UTC (permalink / raw)
To: yocto; +Cc: Fabien Thomas, Yoann Congal
The purpose of this new builder is to report the reproducibility status
of all meta-openembedded recipes layer by layer. It use the same
reproducible selftest than OE-Core but setting only world as target,
and excluding all oecore and other openembedded layer recipes.
Also, the report output directory is split by layers.
Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
---
Changes v1->v2 :
* Split builder into multiple steps, one for each openembedded layers.
* Split reports output in the same way, one for each layers.
* For each step, only to be tested layer and its dependancies are added.
* Every other layers than the one that is tested is excluded from world.
config.json | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 133 insertions(+)
diff --git a/config.json b/config.json
index 05c6794..90762f9 100644
--- a/config.json
+++ b/config.json
@@ -264,6 +264,136 @@
}
},
+ "reproducible-meta-openembedded" : {
+ "MACHINE" : "qemux86-64",
+ "SDKMACHINE" : "x86_64",
+ "DISTRO" : "None",
+ "NEEDREPOS" : ["oecore", "bitbake", "meta-openembedded"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-selftest"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-core = '1'",
+ "EXCLUDE_FROM_WORLD:layer-selftest = '1'",
+ "OEQA_REPRODUCIBLE_TEST_TARGET = 'world'"
+ ],
+ "step1" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-filesystems layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-filesystems/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
+ ]
+ },
+ "step2" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-gnome layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-gnome/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-gnome",
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-python"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-networking-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-meta-python = '1'"
+ ]
+ },
+ "step3" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-initramfs layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-initramfs/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-initramfs"
+ ]
+ },
+ "step4" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-multimedia layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-multimedia/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-multimedia",
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-meta-python = '1'"
+ ]
+ },
+ "step5" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-networking layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-networking/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
+ ]
+ },
+ "step6" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-oe layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-oe/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ]
+ },
+ "step7" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-perl layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-perl/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-perl",
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
+ ]
+ },
+ "step8" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-python layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-python/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
+ ]
+ },
+ "step9" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-webserver layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-webserver/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-webserver",
+ "${BUILDDIR}/../meta-openembedded/meta-oe"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
+ ]
+ },
+ "step10" : {
+ "shortname" : "Reproducible Selftest for openembedded meta-xfce layer",
+ "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-xfce/ DISPLAY=:1 oe-selftest -r reproducible"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-xfce",
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-gnome",
+ "${BUILDDIR}/../meta-openembedded/meta-multimedia",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking"
+ ],
+ "extravars" : [
+ "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-gnome-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-multimedia-layer = '1'",
+ "EXCLUDE_FROM_WORLD:layer-meta-python = '1'",
+ "EXCLUDE_FROM_WORLD:layer-networking-layer = '1'"
+ ]
+ }
+ },
"trigger-build" : {
"SDKMACHINE" : "x86_64",
"MACHINE" : "qemux86-64",
@@ -955,6 +1085,9 @@
"reproducible" : {
"TEMPLATE" : "reproducible"
},
+ "reproducible-meta-openembedded" : {
+ "TEMPLATE" : "reproducible-meta-openembedded"
+ },
"reproducible-ubuntu" : {
"TEMPLATE" : "reproducible"
},
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
2023-10-02 14:54 [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build Fabien Thomas
@ 2023-10-02 17:16 ` Khem Raj
2023-10-05 16:42 ` Richard Purdie
[not found] ` <178B43BD6DF66AF4.3256@lists.yoctoproject.org>
2 siblings, 0 replies; 9+ messages in thread
From: Khem Raj @ 2023-10-02 17:16 UTC (permalink / raw)
To: Fabien Thomas; +Cc: yocto, Yoann Congal
looks fine to me Thanks
On Mon, Oct 2, 2023 at 7:56 AM Fabien Thomas <fabien.thomas@smile.fr> wrote:
>
> The purpose of this new builder is to report the reproducibility status
> of all meta-openembedded recipes layer by layer. It use the same
> reproducible selftest than OE-Core but setting only world as target,
> and excluding all oecore and other openembedded layer recipes.
> Also, the report output directory is split by layers.
>
> Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
> Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
> ---
>
> Changes v1->v2 :
> * Split builder into multiple steps, one for each openembedded layers.
> * Split reports output in the same way, one for each layers.
> * For each step, only to be tested layer and its dependancies are added.
> * Every other layers than the one that is tested is excluded from world.
>
> config.json | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 133 insertions(+)
>
> diff --git a/config.json b/config.json
> index 05c6794..90762f9 100644
> --- a/config.json
> +++ b/config.json
> @@ -264,6 +264,136 @@
>
> }
> },
> + "reproducible-meta-openembedded" : {
> + "MACHINE" : "qemux86-64",
> + "SDKMACHINE" : "x86_64",
> + "DISTRO" : "None",
> + "NEEDREPOS" : ["oecore", "bitbake", "meta-openembedded"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-selftest"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-core = '1'",
> + "EXCLUDE_FROM_WORLD:layer-selftest = '1'",
> + "OEQA_REPRODUCIBLE_TEST_TARGET = 'world'"
> + ],
> + "step1" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-filesystems layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-filesystems/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-filesystems",
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
> + ]
> + },
> + "step2" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-gnome layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-gnome/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-gnome",
> + "${BUILDDIR}/../meta-openembedded/meta-oe",
> + "${BUILDDIR}/../meta-openembedded/meta-networking",
> + "${BUILDDIR}/../meta-openembedded/meta-python"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-networking-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-meta-python = '1'"
> + ]
> + },
> + "step3" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-initramfs layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-initramfs/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-initramfs"
> + ]
> + },
> + "step4" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-multimedia layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-multimedia/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-multimedia",
> + "${BUILDDIR}/../meta-openembedded/meta-oe",
> + "${BUILDDIR}/../meta-openembedded/meta-python"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-meta-python = '1'"
> + ]
> + },
> + "step5" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-networking layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-networking/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-networking",
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
> + ]
> + },
> + "step6" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-oe layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-oe/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ]
> + },
> + "step7" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-perl layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-perl/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-perl",
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
> + ]
> + },
> + "step8" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-python layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-python/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-python",
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
> + ]
> + },
> + "step9" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-webserver layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-webserver/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-webserver",
> + "${BUILDDIR}/../meta-openembedded/meta-oe"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'"
> + ]
> + },
> + "step10" : {
> + "shortname" : "Reproducible Selftest for openembedded meta-xfce layer",
> + "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded-meta-xfce/ DISPLAY=:1 oe-selftest -r reproducible"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-xfce",
> + "${BUILDDIR}/../meta-openembedded/meta-oe",
> + "${BUILDDIR}/../meta-openembedded/meta-gnome",
> + "${BUILDDIR}/../meta-openembedded/meta-multimedia",
> + "${BUILDDIR}/../meta-openembedded/meta-python",
> + "${BUILDDIR}/../meta-openembedded/meta-networking"
> + ],
> + "extravars" : [
> + "EXCLUDE_FROM_WORLD:layer-openembedded-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-gnome-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-multimedia-layer = '1'",
> + "EXCLUDE_FROM_WORLD:layer-meta-python = '1'",
> + "EXCLUDE_FROM_WORLD:layer-networking-layer = '1'"
> + ]
> + }
> + },
> "trigger-build" : {
> "SDKMACHINE" : "x86_64",
> "MACHINE" : "qemux86-64",
> @@ -955,6 +1085,9 @@
> "reproducible" : {
> "TEMPLATE" : "reproducible"
> },
> + "reproducible-meta-openembedded" : {
> + "TEMPLATE" : "reproducible-meta-openembedded"
> + },
> "reproducible-ubuntu" : {
> "TEMPLATE" : "reproducible"
> },
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> You automatically follow any topics you start or reply to.
> View/Reply Online (#61193): https://lists.yoctoproject.org/g/yocto/message/61193
> Mute This Topic: https://lists.yoctoproject.org/mt/101713575/1997914
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
2023-10-02 14:54 [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build Fabien Thomas
2023-10-02 17:16 ` [yocto] " Khem Raj
@ 2023-10-05 16:42 ` Richard Purdie
[not found] ` <178B43BD6DF66AF4.3256@lists.yoctoproject.org>
2 siblings, 0 replies; 9+ messages in thread
From: Richard Purdie @ 2023-10-05 16:42 UTC (permalink / raw)
To: Fabien Thomas, yocto; +Cc: Yoann Congal
On Mon, 2023-10-02 at 16:54 +0200, Fabien Thomas wrote:
> The purpose of this new builder is to report the reproducibility status
> of all meta-openembedded recipes layer by layer. It use the same
> reproducible selftest than OE-Core but setting only world as target,
> and excluding all oecore and other openembedded layer recipes.
> Also, the report output directory is split by layers.
>
> Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
> Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
> ---
>
> Changes v1->v2 :
> * Split builder into multiple steps, one for each openembedded layers.
> * Split reports output in the same way, one for each layers.
> * For each step, only to be tested layer and its dependancies are added.
> * Every other layers than the one that is tested is excluded from world.
Thanks, I've merged this with a tweak of "reproducible-meta-
openembedded" -> "reproducible-meta-oe" just to keep things more
readable in the UI.
There is a test run queued here:
https://autobuilder.yoctoproject.org/typhoon/#/buildrequests/454516?redirect_to_build=true
but things are a bit busy atm on the autobuilder so it is queued.
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread[parent not found: <178B43BD6DF66AF4.3256@lists.yoctoproject.org>]
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
[not found] ` <178B43BD6DF66AF4.3256@lists.yoctoproject.org>
@ 2023-10-05 19:46 ` Richard Purdie
[not found] ` <178B4DC19A31CF35.3256@lists.yoctoproject.org>
1 sibling, 0 replies; 9+ messages in thread
From: Richard Purdie @ 2023-10-05 19:46 UTC (permalink / raw)
To: Fabien Thomas, yocto; +Cc: Yoann Congal
On Thu, 2023-10-05 at 17:42 +0100, Richard Purdie via
lists.yoctoproject.org wrote:
> On Mon, 2023-10-02 at 16:54 +0200, Fabien Thomas wrote:
> > The purpose of this new builder is to report the reproducibility status
> > of all meta-openembedded recipes layer by layer. It use the same
> > reproducible selftest than OE-Core but setting only world as target,
> > and excluding all oecore and other openembedded layer recipes.
> > Also, the report output directory is split by layers.
> >
> > Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
> > Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
> > ---
> >
> > Changes v1->v2 :
> > * Split builder into multiple steps, one for each openembedded layers.
> > * Split reports output in the same way, one for each layers.
> > * For each step, only to be tested layer and its dependancies are added.
> > * Every other layers than the one that is tested is excluded from world.
>
> Thanks, I've merged this with a tweak of "reproducible-meta-
> openembedded" -> "reproducible-meta-oe" just to keep things more
> readable in the UI.
>
> There is a test run queued here:
>
> https://autobuilder.yoctoproject.org/typhoon/#/buildrequests/454516?redirect_to_build=true
>
> but things are a bit busy atm on the autobuilder so it is queued.
That broke as did the second attempt. I've pushed controller side
tweaks and some tweaks to helper which means this is now running:
https://autobuilder.yoctoproject.org/typhoon/#/builders/155/builds/3
Obviously there are still some failures but it is doing more this time
and a least running something in some steps.
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread[parent not found: <178B4DC19A31CF35.3256@lists.yoctoproject.org>]
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
[not found] ` <178B4DC19A31CF35.3256@lists.yoctoproject.org>
@ 2023-10-05 21:51 ` Richard Purdie
2023-10-06 14:27 ` Fabien THOMAS
0 siblings, 1 reply; 9+ messages in thread
From: Richard Purdie @ 2023-10-05 21:51 UTC (permalink / raw)
To: Fabien Thomas, yocto; +Cc: Yoann Congal
On Thu, 2023-10-05 at 20:46 +0100, Richard Purdie via
lists.yoctoproject.org wrote:
> On Thu, 2023-10-05 at 17:42 +0100, Richard Purdie via
> lists.yoctoproject.org wrote:
> > On Mon, 2023-10-02 at 16:54 +0200, Fabien Thomas wrote:
> > > The purpose of this new builder is to report the reproducibility status
> > > of all meta-openembedded recipes layer by layer. It use the same
> > > reproducible selftest than OE-Core but setting only world as target,
> > > and excluding all oecore and other openembedded layer recipes.
> > > Also, the report output directory is split by layers.
> > >
> > > Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
> > > Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
> > > ---
> > >
> > > Changes v1->v2 :
> > > * Split builder into multiple steps, one for each openembedded layers.
> > > * Split reports output in the same way, one for each layers.
> > > * For each step, only to be tested layer and its dependancies are added.
> > > * Every other layers than the one that is tested is excluded from world.
> >
> > Thanks, I've merged this with a tweak of "reproducible-meta-
> > openembedded" -> "reproducible-meta-oe" just to keep things more
> > readable in the UI.
> >
> > There is a test run queued here:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/buildrequests/454516?redirect_to_build=true
> >
> > but things are a bit busy atm on the autobuilder so it is queued.
>
> That broke as did the second attempt. I've pushed controller side
> tweaks and some tweaks to helper which means this is now running:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/155/builds/3
>
> Obviously there are still some failures but it is doing more this time
> and a least running something in some steps.
Whilst a lot of it failed, there was one piece that succeeded and
generated diffoscope output:
https://autobuilder.yoctoproject.org/typhoon/#/builders/155/builds/3/steps/21/logs/stdio
http://autobuilder.yocto.io/pub/repro-fail-openembedded-meta-initramfs/oe-reproducible-20231005-y6zo9rio/packages/diff-html/
As well as fixing the failures, the "short" names need work to make the
UI readable.
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
2023-10-05 21:51 ` Richard Purdie
@ 2023-10-06 14:27 ` Fabien THOMAS
2023-10-11 8:38 ` Alexander Kanavin
0 siblings, 1 reply; 9+ messages in thread
From: Fabien THOMAS @ 2023-10-06 14:27 UTC (permalink / raw)
To: Richard Purdie; +Cc: yocto, Yoann Congal
Thank you Richard for trying this patch, this report is very helpful.
-> Well noted for the short name, indeed it's absolutely not readable
in the UI, I will go with this name "Repro test OE meta-xxx" if you
agree.
-> Reading the logs, I see that all "Add layers" steps always fail
except meta-initramfs and meta-oe, both has no dependencies.
I understand how it works now ! I had to respect the dependencies
order in the "ADDLAYER" list because it uses the bitbake-layers
add-layer cmd.
It will be fixed too.
-> On the build step "Run cmds" for meta-oe, I see following error :
ERROR - Build directory
/home/pokybuild/yocto-worker/reproducible-meta-oe/build/build-st
already exists, aborting
Probably due to previous step failed. Do you think it is necessary to
add an rm -rf [...]/build-st in each step, before actually running
selftest ?
-> Also, I would like to inform you that I'm having build problems
with every layer in the reproducible sefltest context, except with
meta-initramfs.
So I focused myself on meta-oe.
One of the building issues is regarding recipes that use the useradd
class. In the reproducible context, these must use static-id.
I fixed it by creating new files/static-passwd and files/static-group
into meta-oe layer. I plan to send patches to both
yocto-autobuilder-helper & meta-openmebdded project soon.
Also I had trouble with 4 recipes (webkitgtk3_2.40.5, influxdb_1.8.10,
xorgxrdp_0.9.19 and xscreensaver_6.04). I didn't investigate but only
excluded them from the world target.
So my patch might look like this :
"step6" : {
"shortname" : "Repro test OE meta-oe",
"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc;
OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail-openembedded/meta-oe/
DISPLAY=:1 oe-selftest -r reproducible"],
"ADDLAYER" : [
"${BUILDDIR}/../meta-openembedded/meta-oe"
],
"extravars" : [
"USERADD_UID_TABLES:append = '
files/static-passwd-meta-oe'",
"USERADD_GID_TABLES:append = ' files/static-group-meta-oe'"
"EXCLUDE_FROM_WORLD:pn-xscreensaver"
"EXCLUDE_FROM_WORLD:pn-webkitgtk3"
"EXCLUDE_FROM_WORLD:pn-influxdb"
"EXCLUDE_FROM_WORLD:pn-xorgxrdp"
]
},
What do you think ? (Note that for layers with dependencies all
static-* files should be added)
-> With these adjustments, I finally had a successful build on my
computer. FYI here are the results for meta-oe :
oe-selftest - INFO - Reproducibility summary for deb: same=14650
different=128 different_excluded=0 missing=0 total=14778
Le jeu. 5 oct. 2023 à 23:51, Richard Purdie
<richard.purdie@linuxfoundation.org> a écrit :
>
> On Thu, 2023-10-05 at 20:46 +0100, Richard Purdie via
> lists.yoctoproject.org wrote:
> > On Thu, 2023-10-05 at 17:42 +0100, Richard Purdie via
> > lists.yoctoproject.org wrote:
> > > On Mon, 2023-10-02 at 16:54 +0200, Fabien Thomas wrote:
> > > > The purpose of this new builder is to report the reproducibility status
> > > > of all meta-openembedded recipes layer by layer. It use the same
> > > > reproducible selftest than OE-Core but setting only world as target,
> > > > and excluding all oecore and other openembedded layer recipes.
> > > > Also, the report output directory is split by layers.
> > > >
> > > > Signed-off-by: Fabien Thomas <fabien.thomas@smile.fr>
> > > > Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
> > > > ---
> > > >
> > > > Changes v1->v2 :
> > > > * Split builder into multiple steps, one for each openembedded layers.
> > > > * Split reports output in the same way, one for each layers.
> > > > * For each step, only to be tested layer and its dependancies are added.
> > > > * Every other layers than the one that is tested is excluded from world.
> > >
> > > Thanks, I've merged this with a tweak of "reproducible-meta-
> > > openembedded" -> "reproducible-meta-oe" just to keep things more
> > > readable in the UI.
> > >
> > > There is a test run queued here:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/buildrequests/454516?redirect_to_build=true
> > >
> > > but things are a bit busy atm on the autobuilder so it is queued.
> >
> > That broke as did the second attempt. I've pushed controller side
> > tweaks and some tweaks to helper which means this is now running:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/155/builds/3
> >
> > Obviously there are still some failures but it is doing more this time
> > and a least running something in some steps.
>
> Whilst a lot of it failed, there was one piece that succeeded and
> generated diffoscope output:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/155/builds/3/steps/21/logs/stdio
> http://autobuilder.yocto.io/pub/repro-fail-openembedded-meta-initramfs/oe-reproducible-20231005-y6zo9rio/packages/diff-html/
>
> As well as fixing the failures, the "short" names need work to make the
> UI readable.
>
> Cheers,
>
> Richard
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
2023-10-06 14:27 ` Fabien THOMAS
@ 2023-10-11 8:38 ` Alexander Kanavin
2023-10-11 9:26 ` Jose Quaresma
0 siblings, 1 reply; 9+ messages in thread
From: Alexander Kanavin @ 2023-10-11 8:38 UTC (permalink / raw)
To: Fabien Thomas; +Cc: Richard Purdie, yocto, Yoann Congal
On Fri, 6 Oct 2023 at 16:27, Fabien Thomas <fabien.thomas@smile.fr> wrote:
> -> On the build step "Run cmds" for meta-oe, I see following error :
> ERROR - Build directory
> /home/pokybuild/yocto-worker/reproducible-meta-oe/build/build-st
> already exists, aborting
> Probably due to previous step failed. Do you think it is necessary to
> add an rm -rf [...]/build-st in each step, before actually running
> selftest ?
No, because this complicates investigating failures after they
happened. The correct thing to do is to stop the whole job when a
specific step fails. Letting subsequent steps fail due to the already
existing failed directory is ok too.
Alex
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build
2023-10-11 8:38 ` Alexander Kanavin
@ 2023-10-11 9:26 ` Jose Quaresma
2023-10-13 14:41 ` Fabien Thomas
0 siblings, 1 reply; 9+ messages in thread
From: Jose Quaresma @ 2023-10-11 9:26 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: Fabien Thomas, Richard Purdie, yocto, Yoann Congal
[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]
Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia quarta,
11/10/2023 à(s) 09:38:
> On Fri, 6 Oct 2023 at 16:27, Fabien Thomas <fabien.thomas@smile.fr> wrote:
> > -> On the build step "Run cmds" for meta-oe, I see following error :
> > ERROR - Build directory
> > /home/pokybuild/yocto-worker/reproducible-meta-oe/build/build-st
> > already exists, aborting
> > Probably due to previous step failed. Do you think it is necessary to
> > add an rm -rf [...]/build-st in each step, before actually running
> > selftest ?
>
> No, because this complicates investigating failures after they
> happened. The correct thing to do is to stop the whole job when a
> specific step fails. Letting subsequent steps fail due to the already
> existing failed directory is ok too.
>
Other options can be using oe-selftest --newbuilddir
[...]/build-st-${LAYER} and build each layer in your own directory.
I have used this --newbuilddir in my reproducible build and it works.
Jose
>
> Alex
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#61316):
> https://lists.yoctoproject.org/g/yocto/message/61316
> Mute This Topic: https://lists.yoctoproject.org/mt/101713575/5052612
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
--
Best regards,
José Quaresma
[-- Attachment #2: Type: text/html, Size: 2679 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-10-13 14:41 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-02 14:54 [yocto-autobuilder-helper][PATCH v2] config.json: add reproducible-openembedded build Fabien Thomas
2023-10-02 17:16 ` [yocto] " Khem Raj
2023-10-05 16:42 ` Richard Purdie
[not found] ` <178B43BD6DF66AF4.3256@lists.yoctoproject.org>
2023-10-05 19:46 ` Richard Purdie
[not found] ` <178B4DC19A31CF35.3256@lists.yoctoproject.org>
2023-10-05 21:51 ` Richard Purdie
2023-10-06 14:27 ` Fabien THOMAS
2023-10-11 8:38 ` Alexander Kanavin
2023-10-11 9:26 ` Jose Quaresma
2023-10-13 14:41 ` Fabien Thomas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).