[PATCH 1/3] insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

[PATCH 1/3] insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

[Peter Kjellerstedt]

Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to
rerun to make sure any already existing fuzz is caught. This is
achieved by using bb.utils.filter() to see if "patch-fuzz" is in
ERROR_QA/WARN_QA as it adds whether the filtered strings are set or
not to the task hash.

Since the mechanism used above ignores that a variable is excluded
(which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA
or not has to be explicitly excluded so that adding/removing
"patch-fuzz" to/from WARN_QA does not trigger the patch tasks to
rerun.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/insane.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 0deebdb148..873706952b 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1142,6 +1142,8 @@ python do_qa_staging() {
     oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
 }
 
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
+do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"
 python do_qa_patch() {
     import subprocess
 
@@ -1182,9 +1184,9 @@ python do_qa_patch() {
             msg += "    devtool modify %s\n" % d.getVar('PN')
             msg += "    devtool finish --force-patch-refresh %s <layer_path>\n\n" % d.getVar('PN')
             msg += "Don't forget to review changes done by devtool!\n"
-            if 'patch-fuzz' in d.getVar('ERROR_QA'):
+            if bb.utils.filter('ERROR_QA', 'patch-fuzz', d):
                 bb.error(msg)
-            elif 'patch-fuzz' in d.getVar('WARN_QA'):
+            elif bb.utils.filter('WARN_QA', 'patch-fuzz', d):
                 bb.warn(msg)
             msg = "Patch log indicates that patches do not apply cleanly."
             oe.qa.handle_error("patch-fuzz", msg, d)

[PATCH 2/3] bitbake.conf: Remove ERROR_QA from BB_HASHEXCLUDE_COMMON

[Peter Kjellerstedt]

If a QA test is added to ERROR_QA, all package_qa tasks should rerun
to make sure any already existing errors are caught.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 7705415a4f..3b8760a3f8 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -949,7 +949,7 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
     USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
     STAMPS_DIR PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
     CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_NOHASHDIR LICENSE_PATH SDKPKGSUFFIX \
-    WARN_QA ERROR_QA WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
+    WARN_QA WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
     BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH extend_recipe_sysroot DEPLOY_DIR \
     SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
     SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \

[PATCH 3/3] insane.bbclass: Make changes to QA_EMPTY_DIRS trigger package_qa to rerun

[Peter Kjellerstedt]

If "empty-dirs" is in ERROR_QA, then changes to QA_EMPTY_DIRS should
trigger the package_qa tasks to be rerun.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/insane.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 873706952b..ca67a70c43 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -1117,6 +1117,7 @@ python do_package_qa () {
 # binutils is used for most checks, so need to set as dependency
 # POPULATESYSROOTDEPS is defined in staging class.
 do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
+do_package_qa[vardeps] = "${@bb.utils.contains('ERROR_QA', 'empty-dirs', 'QA_EMPTY_DIRS', '', d)}"
 do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
 do_package_qa[rdeptask] = "do_packagedata"
 addtask do_package_qa after do_packagedata do_package before do_build

Re: [OE-core] [PATCH 1/3] insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

[Richard Purdie]

On Sat, 2022-04-02 at 01:59 +0200, Peter Kjellerstedt wrote:
> Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to
> rerun to make sure any already existing fuzz is caught. This is
> achieved by using bb.utils.filter() to see if "patch-fuzz" is in
> ERROR_QA/WARN_QA as it adds whether the filtered strings are set or
> not to the task hash.
> 
> Since the mechanism used above ignores that a variable is excluded
> (which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA
> or not has to be explicitly excluded so that adding/removing
> "patch-fuzz" to/from WARN_QA does not trigger the patch tasks to
> rerun.
> 
> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> ---
>  meta/classes/insane.bbclass | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index 0deebdb148..873706952b 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -1142,6 +1142,8 @@ python do_qa_staging() {
>      oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
>  }
>  
> +do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
> +do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"

I'd never seen that syntax before, it certainly wasn't intentional. Looking at
the code, I can see how it happens to work though. It does assume a bit more
internal implementation knowledge than I'd prefer...

Cheers,

Richard

Re: [OE-core] [PATCH 1/3] insane.bbclass: Make do_qa_patch() depend on if patch-fuzz is in ERROR_QA

[Richard Purdie]

On Sat, 2022-04-02 at 07:52 +0100, Richard Purdie via lists.openembedded.org
wrote:
> On Sat, 2022-04-02 at 01:59 +0200, Peter Kjellerstedt wrote:
> > Adding "patch-fuzz" to ERROR_QA should trigger the patch tasks to
> > rerun to make sure any already existing fuzz is caught. This is
> > achieved by using bb.utils.filter() to see if "patch-fuzz" is in
> > ERROR_QA/WARN_QA as it adds whether the filtered strings are set or
> > not to the task hash.
> > 
> > Since the mechanism used above ignores that a variable is excluded
> > (which WARN_QA is), the value for whether "patch-fuzz" is in WARN_QA
> > or not has to be explicitly excluded so that adding/removing
> > "patch-fuzz" to/from WARN_QA does not trigger the patch tasks to
> > rerun.
> > 
> > Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> > ---
> >  meta/classes/insane.bbclass | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> > index 0deebdb148..873706952b 100644
> > --- a/meta/classes/insane.bbclass
> > +++ b/meta/classes/insane.bbclass
> > @@ -1142,6 +1142,8 @@ python do_qa_staging() {
> >      oe.qa.exit_with_message_if_errors("QA staging was broken by the package built above", d)
> >  }
> >  
> > +do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Set"
> > +do_qa_patch[vardepvalueexclude] .= "|WARN_QA{patch-fuzz} = Unset"
> 
> I'd never seen that syntax before, it certainly wasn't intentional. Looking at
> the code, I can see how it happens to work though. It does assume a bit more
> internal implementation knowledge than I'd prefer...

I've a patch I'm testing in master-next to try and address this and remove the
need for this piece of patch.

I need to be a little careful as the loop in question is one of the most time
sensitive we have in the parsing code being iterated millions of times, so
changes need to be mindful of that.

Cheers,

Richard