* [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test @ 2023-10-20 14:05 David Pierret 2023-10-20 14:57 ` [yocto] " Richard Purdie 0 siblings, 1 reply; 5+ messages in thread From: David Pierret @ 2023-10-20 14:05 UTC (permalink / raw) To: yocto; +Cc: David Pierret - Add setup and run script openembedded specific - Add job config Signed-off-by: David Pierret <david.pierret@smile.fr> --- config.json | 6 ++++++ scripts/run-auh-oe | 45 ++++++++++++++++++++++++++++++++++++++++++++ scripts/setup-auh-oe | 34 +++++++++++++++++++++++++++++++++ 3 files changed, 85 insertions(+) create mode 100755 scripts/run-auh-oe create mode 100755 scripts/setup-auh-oe diff --git a/config.json b/config.json index 3acb710..bbd0aaf 100644 --- a/config.json +++ b/config.json @@ -1420,6 +1420,12 @@ "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/" ] }, + "auh-openembedded" : { + "NEEDREPOS" : ["poky", "meta-openembedded"], + "EXTRAPLAINCMDS" : [ + "${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded ${HELPERBUILDDIR}/meta-openembedded/meta-oe ${HELPERBUILDDIR}/meta-openembedded/meta-python ${HELPERBUILDDIR}/meta-openembedded/meta-perl ${HELPERBUILDDIR}/meta-openembedded/meta-networking ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia ${HELPERBUILDDIR}/meta-openembedded/meta-gnome ${HELPERBUILDDIR}/meta-openembedded/meta-xfce ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs ${HELPERBUILDDIR}/meta-openembedded/meta-webserver " + ] + }, "a-quick" : { "TEMPLATE" : "trigger-build" }, diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe new file mode 100755 index 0000000..588755a --- /dev/null +++ b/scripts/run-auh-oe @@ -0,0 +1,45 @@ +#!/bin/bash +# +# SPDX-License-Identifier: GPL-2.0-only +# +# Run Auto Upgrade Helper in a directory set up by setup_auh. +# +# Called with $1 - the directory where the setup was created + +if [ -z $1 ]; then + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]" + exit 1 +fi + +full_dir=$(readlink -e $1) + +auh_dir=$full_dir/auto-upgrade-helper +poky_dir=$full_dir/poky + +build_dir=$full_dir/build +sstate_dir=$full_dir/build/sstate-cache +meta_dir=$3 +meta_list=${@:4} +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl" + +pushd $meta_dir || exit 1 + +# Base the upgrades on meta_openembedded master +git fetch origin +git checkout -B tmp-auh-upgrades origin/main + +source $poky_dir/oe-init-build-env $build_dir + +# build the layer_names variable to me used in the command line +layer_names="" +for d in $meta_list; do + layer_names+=" $(basename ${d})" +done + +$auh_dir/upgrade-helper.py -e --layer-dir ${meta_dir} --layer-names ${layer_names} --layer-machines ${machine_list} -- all + +if [ -n $2 ]; then + cp -rf $build_dir/upgrade-helper/* $2 +fi + +popd diff --git a/scripts/setup-auh-oe b/scripts/setup-auh-oe new file mode 100755 index 0000000..ba1a7fb --- /dev/null +++ b/scripts/setup-auh-oe @@ -0,0 +1,34 @@ +#!/bin/bash +# +# SPDX-License-Identifier: GPL-2.0-only +# +# Initialize Auto Upgrade Helper in a directory. +# +# Called with $1 - the directory to place the setup +CONFIG_DIR=`dirname $0`/auh-config + +if [ -z $1 ]; then + echo "Use: $0 target_dir" + exit 1 +fi + +mkdir -p $1 +pushd $1 + +git clone git://git.yoctoproject.org/poky +pushd poky +git config user.email auh@yoctoproject.org +git config user.name "Auto Upgrade Helper" +popd +git clone git://git.openembedded.org/meta-openembedded +pushd meta-openembedded +git config user.email auh@yoctoproject.org +git config user.name "Auto Upgrade Helper" +popd +git clone git://git.yoctoproject.org/auto-upgrade-helper +source poky/oe-init-build-env build +mkdir -p upgrade-helper +popd + +cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper +cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf -- 2.39.2 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test 2023-10-20 14:05 [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test David Pierret @ 2023-10-20 14:57 ` Richard Purdie 2023-10-24 8:16 ` David PIERRET 0 siblings, 1 reply; 5+ messages in thread From: Richard Purdie @ 2023-10-20 14:57 UTC (permalink / raw) To: David Pierret, yocto; +Cc: Khem Raj Hi David, Thanks for revising this, it is heading in the right direction. On Fri, 2023-10-20 at 16:05 +0200, David Pierret wrote: > - Add setup and run script openembedded specific > - Add job config > > Signed-off-by: David Pierret <david.pierret@smile.fr> > --- > config.json | 6 ++++++ > scripts/run-auh-oe | 45 ++++++++++++++++++++++++++++++++++++++++++++ > scripts/setup-auh-oe | 34 +++++++++++++++++++++++++++++++++ > 3 files changed, 85 insertions(+) > create mode 100755 scripts/run-auh-oe > create mode 100755 scripts/setup-auh-oe > > diff --git a/config.json b/config.json > index 3acb710..bbd0aaf 100644 > --- a/config.json > +++ b/config.json > @@ -1420,6 +1420,12 @@ > "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/" > ] > }, > + "auh-openembedded" : { This needs to be "auh-meta-oe" since "auh" is for openembedded-core and this is otherwise going to be confusing. > + "NEEDREPOS" : ["poky", "meta-openembedded"], > + "EXTRAPLAINCMDS" : [ > + "${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded ${HELPERBUILDDIR}/meta-openembedded/meta-oe ${HELPERBUILDDIR}/meta-openembedded/meta-python ${HELPERBUILDDIR}/meta-openembedded/meta-perl ${HELPERBUILDDIR}/meta-openembedded/meta-networking ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia ${HELPERBUILDDIR}/meta-openembedded/meta-gnome ${HELPERBUILDDIR}/meta-openembedded/meta-xfce ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs ${HELPERBUILDDIR}/meta-openembedded/meta-webserver " > + ] > + }, > "a-quick" : { > "TEMPLATE" : "trigger-build" > }, Would it be better to have one setup-auh/run-auh script and add ${HELPERTARGET} as a parameter? This should translate to "auh" or "auh-meta-oe" and the script can then have conditionals in to handle both cases rather than duplicating the script? > diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe > new file mode 100755 > index 0000000..588755a > --- /dev/null > +++ b/scripts/run-auh-oe Similarly, this should be run-auh-meta-oe. > @@ -0,0 +1,45 @@ > +#!/bin/bash > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# Run Auto Upgrade Helper in a directory set up by setup_auh. > +# > +# Called with $1 - the directory where the setup was created > + > +if [ -z $1 ]; then > + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]" > + exit 1 > +fi > + > +full_dir=$(readlink -e $1) > + > +auh_dir=$full_dir/auto-upgrade-helper > +poky_dir=$full_dir/poky > + > +build_dir=$full_dir/build > +sstate_dir=$full_dir/build/sstate-cache > +meta_dir=$3 > +meta_list=${@:4} > +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl" Do we need to vary the machine_list per layer? At present it only seems to support one list anyway so I'm not sure this does anything useful? > +pushd $meta_dir || exit 1 > + > +# Base the upgrades on meta_openembedded master > +git fetch origin > +git checkout -B tmp-auh-upgrades origin/main > + > +source $poky_dir/oe-init-build-env $build_dir > + > +# build the layer_names variable to me used in the command line > +layer_names="" > +for d in $meta_list; do > + layer_names+=" $(basename ${d})" > +done > + > +$auh_dir/upgrade-helper.py -e --layer-dir ${meta_dir} --layer-names ${layer_names} --layer-machines ${machine_list} -- all Would it be simpler just to iterate, calling upgrade-helper once per sub-layer or meta-openembedded? You may have considered that in which case I'd just like to understand why you ended up with this as the preferred way of doing things? In some ways a separate report/run may be useful for the way meta-oe maintainers might handle this? > + > +if [ -n $2 ]; then > + cp -rf $build_dir/upgrade-helper/* $2 > +fi > + > +popd > diff --git a/scripts/setup-auh-oe b/scripts/setup-auh-oe > new file mode 100755 > index 0000000..ba1a7fb > --- /dev/null > +++ b/scripts/setup-auh-oe This naming also needs tweaking. > @@ -0,0 +1,34 @@ > +#!/bin/bash > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# Initialize Auto Upgrade Helper in a directory. > +# > +# Called with $1 - the directory to place the setup > +CONFIG_DIR=`dirname $0`/auh-config > + > +if [ -z $1 ]; then > + echo "Use: $0 target_dir" > + exit 1 > +fi > + > +mkdir -p $1 > +pushd $1 > + > +git clone git://git.yoctoproject.org/poky > +pushd poky > +git config user.email auh@yoctoproject.org > +git config user.name "Auto Upgrade Helper" > +popd > +git clone git://git.openembedded.org/meta-openembedded > +pushd meta-openembedded > +git config user.email auh@yoctoproject.org > +git config user.name "Auto Upgrade Helper" > +popd > +git clone git://git.yoctoproject.org/auto-upgrade-helper > +source poky/oe-init-build-env build > +mkdir -p upgrade-helper > +popd > + > +cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper > +cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf Would a upgrade-helper.conf per target help and simplfy the script differences? Cheers, Richard ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test 2023-10-20 14:57 ` [yocto] " Richard Purdie @ 2023-10-24 8:16 ` David PIERRET 2023-10-24 8:58 ` Alexander Kanavin 2023-10-24 9:25 ` Richard Purdie 0 siblings, 2 replies; 5+ messages in thread From: David PIERRET @ 2023-10-24 8:16 UTC (permalink / raw) To: Richard Purdie; +Cc: yocto, Khem Raj Hi Richard, Thanks for the review, please find my feedback in follow. On Fri, Oct 20, 2023 at 4:57 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > Hi David, > > Thanks for revising this, it is heading in the right direction. > > On Fri, 2023-10-20 at 16:05 +0200, David Pierret wrote: > > - Add setup and run script openembedded specific > > - Add job config > > > > Signed-off-by: David Pierret <david.pierret@smile.fr> > > --- > > config.json | 6 ++++++ > > scripts/run-auh-oe | 45 ++++++++++++++++++++++++++++++++++++++++++++ > > scripts/setup-auh-oe | 34 +++++++++++++++++++++++++++++++++ > > 3 files changed, 85 insertions(+) > > create mode 100755 scripts/run-auh-oe > > create mode 100755 scripts/setup-auh-oe > > > > diff --git a/config.json b/config.json > > index 3acb710..bbd0aaf 100644 > > --- a/config.json > > +++ b/config.json > > @@ -1420,6 +1420,12 @@ > > "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/" > > ] > > }, > > + "auh-openembedded" : { > > This needs to be "auh-meta-oe" since "auh" is for openembedded-core and > this is otherwise going to be confusing. > > > + "NEEDREPOS" : ["poky", "meta-openembedded"], > > + "EXTRAPLAINCMDS" : [ > > + "${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded ${HELPERBUILDDIR}/meta-openembedded/meta-oe ${HELPERBUILDDIR}/meta-openembedded/meta-python ${HELPERBUILDDIR}/meta-openembedded/meta-perl ${HELPERBUILDDIR}/meta-openembedded/meta-networking ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia ${HELPERBUILDDIR}/meta-openembedded/meta-gnome ${HELPERBUILDDIR}/meta-openembedded/meta-xfce ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs ${HELPERBUILDDIR}/meta-openembedded/meta-webserver " > > + ] > > + }, > > "a-quick" : { > > "TEMPLATE" : "trigger-build" > > }, > > Would it be better to have one setup-auh/run-auh script and add > ${HELPERTARGET} as a parameter? > > This should translate to "auh" or "auh-meta-oe" and the script can then > have conditionals in to handle both cases rather than duplicating the > script? The setup-auh script is cloning its own repository instead of using the one initialized by CI. Any reason to not use them ? We could use the NEEDREPOS instead of the setup script, and add the auto-upgrade-helper repository in the config. > > > diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe > > new file mode 100755 > > index 0000000..588755a > > --- /dev/null > > +++ b/scripts/run-auh-oe > > Similarly, this should be run-auh-meta-oe. > > > @@ -0,0 +1,45 @@ > > +#!/bin/bash > > +# > > +# SPDX-License-Identifier: GPL-2.0-only > > +# > > +# Run Auto Upgrade Helper in a directory set up by setup_auh. > > +# > > +# Called with $1 - the directory where the setup was created > > + > > +if [ -z $1 ]; then > > + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]" > > + exit 1 > > +fi > > + > > +full_dir=$(readlink -e $1) > > + > > +auh_dir=$full_dir/auto-upgrade-helper > > +poky_dir=$full_dir/poky > > + > > +build_dir=$full_dir/build > > +sstate_dir=$full_dir/build/sstate-cache > > +meta_dir=$3 > > +meta_list=${@:4} > > +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl" > > Do we need to vary the machine_list per layer? At present it only seems > to support one list anyway so I'm not sure this does anything useful? The list is the same as the one used for poky. > > > +pushd $meta_dir || exit 1 > > + > > +# Base the upgrades on meta_openembedded master > > +git fetch origin > > +git checkout -B tmp-auh-upgrades origin/main > > + > > +source $poky_dir/oe-init-build-env $build_dir > > + > > +# build the layer_names variable to me used in the command line > > +layer_names="" > > +for d in $meta_list; do > > + layer_names+=" $(basename ${d})" > > +done > > + > > +$auh_dir/upgrade-helper.py -e --layer-dir ${meta_dir} --layer-names ${layer_names} --layer-machines ${machine_list} -- all > > Would it be simpler just to iterate, calling upgrade-helper once per > sub-layer or meta-openembedded? Is it prefered to set 1 step per layer ? or keep the call to the script but iterate over a list of layers ? > > You may have considered that in which case I'd just like to understand > why you ended up with this as the preferred way of doing things? > > In some ways a separate report/run may be useful for the way meta-oe > maintainers might handle this? The iteration over a list was my first approach, but aborted following review on patch v1. The V1 review asked to allow the AUH to accept dynamic layers as parameters. We maybe misunderstood each other. > > > + > > +if [ -n $2 ]; then > > + cp -rf $build_dir/upgrade-helper/* $2 > > +fi > > + > > +popd > > diff --git a/scripts/setup-auh-oe b/scripts/setup-auh-oe > > new file mode 100755 > > index 0000000..ba1a7fb > > --- /dev/null > > +++ b/scripts/setup-auh-oe > > This naming also needs tweaking. > > > @@ -0,0 +1,34 @@ > > +#!/bin/bash > > +# > > +# SPDX-License-Identifier: GPL-2.0-only > > +# > > +# Initialize Auto Upgrade Helper in a directory. > > +# > > +# Called with $1 - the directory to place the setup > > +CONFIG_DIR=`dirname $0`/auh-config > > + > > +if [ -z $1 ]; then > > + echo "Use: $0 target_dir" > > + exit 1 > > +fi > > + > > +mkdir -p $1 > > +pushd $1 > > + > > +git clone git://git.yoctoproject.org/poky > > +pushd poky > > +git config user.email auh@yoctoproject.org > > +git config user.name "Auto Upgrade Helper" > > +popd > > +git clone git://git.openembedded.org/meta-openembedded > > +pushd meta-openembedded > > +git config user.email auh@yoctoproject.org > > +git config user.name "Auto Upgrade Helper" > > +popd > > +git clone git://git.yoctoproject.org/auto-upgrade-helper > > +source poky/oe-init-build-env build > > +mkdir -p upgrade-helper > > +popd > > + > > +cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper > > +cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf > > Would a upgrade-helper.conf per target help and simplfy the script > differences? Do you mean 1 upgrade-helper.conf per layer ? > > Cheers, > > Richard > > regards David ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test 2023-10-24 8:16 ` David PIERRET @ 2023-10-24 8:58 ` Alexander Kanavin 2023-10-24 9:25 ` Richard Purdie 1 sibling, 0 replies; 5+ messages in thread From: Alexander Kanavin @ 2023-10-24 8:58 UTC (permalink / raw) To: David Pierret; +Cc: Richard Purdie, yocto, Khem Raj On Tue, 24 Oct 2023 at 10:17, David Pierret <david.pierret@smile.fr> wrote: > The iteration over a list was my first approach, but aborted following > review on patch v1. > The V1 review asked to allow the AUH to accept dynamic layers as parameters. > We maybe misunderstood each other. I would suggest that AUH changes where dynamic layer configuration is given over command line gets merged first, and this patch is adjusted to use that. Then we can work out how to use that on higher levels.. I also lean towards giving AUH only one layer to work on per invocation, and specific layers being defined in autobuilder configurations. This would allow more efficient autobuilder execution, if layers are worked on in parallel (meta-oe as a whole is huge). Alex ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test 2023-10-24 8:16 ` David PIERRET 2023-10-24 8:58 ` Alexander Kanavin @ 2023-10-24 9:25 ` Richard Purdie 1 sibling, 0 replies; 5+ messages in thread From: Richard Purdie @ 2023-10-24 9:25 UTC (permalink / raw) To: David PIERRET; +Cc: yocto, Khem Raj Hi David, Comments below. I did try and sync up with Alex Kanavin so we have a consistent direction. On Tue, 2023-10-24 at 10:16 +0200, David PIERRET wrote: > On Fri, Oct 20, 2023 at 4:57 PM Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > Hi David, > > > > Thanks for revising this, it is heading in the right direction. > > > > On Fri, 2023-10-20 at 16:05 +0200, David Pierret wrote: > > > - Add setup and run script openembedded specific > > > - Add job config > > > > > > Signed-off-by: David Pierret <david.pierret@smile.fr> > > > --- > > > config.json | 6 ++++++ > > > scripts/run-auh-oe | 45 ++++++++++++++++++++++++++++++++++++++++++++ > > > scripts/setup-auh-oe | 34 +++++++++++++++++++++++++++++++++ > > > 3 files changed, 85 insertions(+) > > > create mode 100755 scripts/run-auh-oe > > > create mode 100755 scripts/setup-auh-oe > > > > > > diff --git a/config.json b/config.json > > > index 3acb710..bbd0aaf 100644 > > > --- a/config.json > > > +++ b/config.json > > > @@ -1420,6 +1420,12 @@ > > > "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/" > > > ] > > > }, > > > + "auh-openembedded" : { > > > > This needs to be "auh-meta-oe" since "auh" is for openembedded-core and > > this is otherwise going to be confusing. > > > > > + "NEEDREPOS" : ["poky", "meta-openembedded"], > > > + "EXTRAPLAINCMDS" : [ > > > + "${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR}; ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded ${HELPERBUILDDIR}/meta-openembedded/meta-oe ${HELPERBUILDDIR}/meta-openembedded/meta-python ${HELPERBUILDDIR}/meta-openembedded/meta-perl ${HELPERBUILDDIR}/meta-openembedded/meta-networking ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia ${HELPERBUILDDIR}/meta-openembedded/meta-gnome ${HELPERBUILDDIR}/meta-openembedded/meta-xfce ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs ${HELPERBUILDDIR}/meta-openembedded/meta-webserver " > > > + ] > > > + }, > > > "a-quick" : { > > > "TEMPLATE" : "trigger-build" > > > }, > > > > Would it be better to have one setup-auh/run-auh script and add > > ${HELPERTARGET} as a parameter? > > > > This should translate to "auh" or "auh-meta-oe" and the script can then > > have conditionals in to handle both cases rather than duplicating the > > script? > > The setup-auh script is cloning its own repository instead of using > the one initialized by CI. > Any reason to not use them ? We could use the NEEDREPOS instead of the > setup script, > and add the auto-upgrade-helper repository in the config. I think it does it's own clone for historical reasons from when it was a standalone tool. I'm fine with letting the autobuilder handle this. > > > diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe > > > new file mode 100755 > > > index 0000000..588755a > > > --- /dev/null > > > +++ b/scripts/run-auh-oe > > > > Similarly, this should be run-auh-meta-oe. > > > > > @@ -0,0 +1,45 @@ > > > +#!/bin/bash > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-only > > > +# > > > +# Run Auto Upgrade Helper in a directory set up by setup_auh. > > > +# > > > +# Called with $1 - the directory where the setup was created > > > + > > > +if [ -z $1 ]; then > > > + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]" > > > + exit 1 > > > +fi > > > + > > > +full_dir=$(readlink -e $1) > > > + > > > +auh_dir=$full_dir/auto-upgrade-helper > > > +poky_dir=$full_dir/poky > > > + > > > +build_dir=$full_dir/build > > > +sstate_dir=$full_dir/build/sstate-cache > > > +meta_dir=$3 > > > +meta_list=${@:4} > > > +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl" > > > > Do we need to vary the machine_list per layer? At present it only seems > > to support one list anyway so I'm not sure this does anything useful? > > The list is the same as the one used for poky. It matches the default in the scripts though so I was wondering why this was specified? FWIW I agree we should perhaps limit testing to qemux86-64, qemuarm and qemux86_musl initially just to make things a little easier. > > > +pushd $meta_dir || exit 1 > > > + > > > +# Base the upgrades on meta_openembedded master > > > +git fetch origin > > > +git checkout -B tmp-auh-upgrades origin/main > > > + > > > +source $poky_dir/oe-init-build-env $build_dir > > > + > > > +# build the layer_names variable to me used in the command line > > > +layer_names="" > > > +for d in $meta_list; do > > > + layer_names+=" $(basename ${d})" > > > +done > > > + > > > +$auh_dir/upgrade-helper.py -e --layer-dir ${meta_dir} --layer-names ${layer_names} --layer-machines ${machine_list} -- all > > > > Would it be simpler just to iterate, calling upgrade-helper once per > > sub-layer or meta-openembedded? > > Is it prefered to set 1 step per layer ? > or keep the call to the script but iterate over a list of layers ? I think a step per layer will make things clearer in the autobuilder output. > > You may have considered that in which case I'd just like to understand > > why you ended up with this as the preferred way of doing things? > > > > In some ways a separate report/run may be useful for the way meta-oe > > maintainers might handle this? > > The iteration over a list was my first approach, but aborted following > review on patch v1. > The V1 review asked to allow the AUH to accept dynamic layers as parameters. > We maybe misunderstood each other. I think this is perhaps a misunderstanding. I think we do want AUH to learn the concept of testing just a layer but we do want to keep the iteration as autobuilder steps to keep the output and logs separate. There are also different maintainers for each layer in meta-oe so we may want to copy different people on different bits of the run, we certainly need that capability. > > > + echo "Use: $0 target_dir" > > > + exit 1 > > > +fi > > > + > > > +mkdir -p $1 > > > +pushd $1 > > > + > > > +git clone git://git.yoctoproject.org/poky > > > +pushd poky > > > +git config user.email auh@yoctoproject.org > > > +git config user.name "Auto Upgrade Helper" > > > +popd > > > +git clone git://git.openembedded.org/meta-openembedded > > > +pushd meta-openembedded > > > +git config user.email auh@yoctoproject.org > > > +git config user.name "Auto Upgrade Helper" > > > +popd > > > +git clone git://git.yoctoproject.org/auto-upgrade-helper > > > +source poky/oe-init-build-env build > > > +mkdir -p upgrade-helper > > > +popd > > > + > > > +cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper > > > +cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf > > > > Would a upgrade-helper.conf per target help and simplfy the script > > differences? > > Do you mean 1 upgrade-helper.conf per layer ? That would allow us to copy different maintainers on different parts of the run for example so might be useful? Also, at least initially can we configure this to send the output to test-list@lists.yoctoproject.org? That way we can test it is working, then make it "live" without spamming the main list. Cheers, Richard ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-10-24 9:25 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-10-20 14:05 [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test David Pierret 2023-10-20 14:57 ` [yocto] " Richard Purdie 2023-10-24 8:16 ` David PIERRET 2023-10-24 8:58 ` Alexander Kanavin 2023-10-24 9:25 ` Richard Purdie
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).