do_rootfs failed ... bug in log_check?

do_rootfs failed ... bug in log_check?

[Oren Leaffer]

[-- Attachment #1: Type: text/plain, Size: 1566 bytes --]

Hi,

I'm getting the following failure on "bitbake -v -D -D -D systemd-image":
ERROR: Task 8 (/home/ol/setup-scripts/sources/meta-angstrom/recipes-images/angstrom/systemd-image.bb,
do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3283 tasks of which 3282 didn't need to
be rerun and 1 failed.
+ cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/bblayers-1.0-r0/bblayers-1.0
+ buildhistory_commit
+ '[' '!' -d /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory
']'
+ cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory/
+ '[' '!' -d .git ']'
++ git status --porcelain
+ repostatus=
+ '[' '' '!=' '' ']'

From what I can tell, this is caused because the log_check command in
setup-scripts/sources/openembedded-core/meta/classes/package_ipk.class
matches on line 177 to

file://3.2.1/0042-xhci-Properly-handle-COMP_2ND_BW_ERR.patch

and to

file://3.2.6/0072-target-Fail-INQUIRY-commands-with-EVPD-0-but-PAGE-CO.patch

It occurs to me that this might be a result of the level of
verbosity/debugging that I specified.

Attached is my patch that does two things:
1) makes the error message from log_check slightly more informative
2) changes the text that log_check is using to identify errors to
incorporate regular expressions

This makes it less likely to issue false positives, I hope that it
doesn't increase the changes of false negatives, but I'm a rank
amateur at this, so I wouldn't bet on it.


Best regards,


Oren

[-- Attachment #2: 0001-making-log_check-less-likely-to-declare-failure-inco.patch --]
[-- Type: application/octet-stream, Size: 2980 bytes --]

From 3fac2bc798607eabce40f611596a4bd895102fb8 Mon Sep 17 00:00:00 2001
From: Oren Leaffer <oren@ephemeron-labs.com>
Date: Fri, 30 Mar 2012 02:23:35 -0400
Subject: [PATCH] making log_check less likely to declare failure incorrectly

---
 meta/classes/package_deb.bbclass |    4 ++--
 meta/classes/package_ipk.bbclass |    6 +++---
 meta/classes/package_rpm.bbclass |    4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 1f7ec9c..78c032e 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -191,8 +191,8 @@ deb_log_check() {
 	do
 		if (echo "$lf_txt" | grep -v log_check | grep "$keyword_die") >/dev/null 2>&1
 		then
-			echo "log_check: There were error messages in the logfile"
-			echo -e "log_check: Matched keyword: [$keyword_die]\n"
+			echo "deb_log_check: There were error messages in the logfile"
+			echo -e "deb_log_check: Matched keyword: [$keyword_die]\n"
 			echo "$lf_txt" | grep -v log_check | grep -C 5 -i "$keyword_die"
 			echo ""
 			do_exit=1
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 565ef93..8c511c1 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -174,12 +174,12 @@ ipk_log_check() {
        lf_path="$2"
 
        lf_txt="`cat $lf_path`"
-       for keyword_die in "exit 1" "Collected errors" ERR Fail
+       for keyword_die in "exit 1" "Collected errors" "^ERR" "^Fail "
        do
                if (echo "$lf_txt" | grep -v log_check | grep "$keyword_die") >/dev/null 2>&1
                then
-                       echo "log_check: There were error messages in the logfile"
-                       echo -e "log_check: Matched keyword: [$keyword_die]\n"
+                       echo "ipk_log_check: There were error messages in the logfile"
+                       echo -e "ipk_log_check: Matched keyword: [$keyword_die]\n"
                        echo "$lf_txt" | grep -v log_check | grep -C 5 -i "$keyword_die"
                        echo ""
                        do_exit=1
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 0ccf574..c014a38 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -119,8 +119,8 @@ rpm_log_check() {
        do
                if (echo "$lf_txt" | grep -v log_check | grep "$keyword_die") >/dev/null 2>&1
                then
-                       echo "log_check: There were error messages in the logfile"
-                       echo -e "log_check: Matched keyword: [$keyword_die]\n"
+                       echo "rpm_log_check: There were error messages in the logfile"
+                       echo -e "rpm_log_check: Matched keyword: [$keyword_die]\n"
                        echo "$lf_txt" | grep -v log_check | grep -C 5 -i "$keyword_die"
                        echo ""
                        do_exit=1
-- 
1.7.0.4

Re: do_rootfs failed ... bug in log_check?

[Denys Dmytriyenko]

On Fri, Mar 30, 2012 at 02:27:13AM -0400, Oren Leaffer wrote:
> Hi,
> 
> I'm getting the following failure on "bitbake -v -D -D -D systemd-image":
> ERROR: Task 8 (/home/ol/setup-scripts/sources/meta-angstrom/recipes-images/angstrom/systemd-image.bb,
> do_rootfs) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 3283 tasks of which 3282 didn't need to
> be rerun and 1 failed.
> + cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/bblayers-1.0-r0/bblayers-1.0
> + buildhistory_commit
> + '[' '!' -d /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory
> ']'
> + cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory/
> + '[' '!' -d .git ']'
> ++ git status --porcelain
> + repostatus=
> + '[' '' '!=' '' ']'

We have been discussing this topic past couple days here:
http://thread.gmane.org/gmane.linux.distributions.angstrom.devel/6260/focus=6274

Moreover, bitbake-devel is probably the wrongest list for this discussion or 
patch. I'm copying other related lists, but probably the main one should be 
openembedded-core, as the patch tries to fix it's package_ipk class...


> From what I can tell, this is caused because the log_check command in
> setup-scripts/sources/openembedded-core/meta/classes/package_ipk.class
> matches on line 177 to
> 
> file://3.2.1/0042-xhci-Properly-handle-COMP_2ND_BW_ERR.patch
> 
> and to
> 
> file://3.2.6/0072-target-Fail-INQUIRY-commands-with-EVPD-0-but-PAGE-CO.patch

And your analysis seems correct, as the grep matches the patch names of 
beaglebone kernel from meta-ti.


> It occurs to me that this might be a result of the level of
> verbosity/debugging that I specified.

Actually, it does not depend on the verbosity/debug level, as it controls the 
output on the console and not in the log.do_rootfs file.


> Attached is my patch that does two things:
> 1) makes the error message from log_check slightly more informative

I don't think this part is really necessary - you always know which packaging 
system you use, so ipk/deb/rpm_log_check is rather easily identifiable...


> 2) changes the text that log_check is using to identify errors to
> incorporate regular expressions

So, this changes the regular expression to look for ^ERR and ^Fail at the 
beginning of the line...


> This makes it less likely to issue false positives, I hope that it
> doesn't increase the changes of false negatives, but I'm a rank
> amateur at this, so I wouldn't bet on it.

While the change should fix this particular issue, I'm also worried about 
increasing chances of missing the actual errors when they don't start with ERR 
and Fail at the beginning of the line.

Either way, you may want to resubmit the cleaned up patch to openembedded-core 
list for other to comment.

P.S. Sorry for somewhat excessive cross-posting.

-- 
Denys

Re: [bitbake-devel] do_rootfs failed ... bug in log_check?

[Denys Dmytriyenko]

On Fri, Mar 30, 2012 at 02:27:13AM -0400, Oren Leaffer wrote:
> Hi,
> 
> I'm getting the following failure on "bitbake -v -D -D -D systemd-image":
> ERROR: Task 8 (/home/ol/setup-scripts/sources/meta-angstrom/recipes-images/angstrom/systemd-image.bb,
> do_rootfs) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 3283 tasks of which 3282 didn't need to
> be rerun and 1 failed.
> + cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/bblayers-1.0-r0/bblayers-1.0
> + buildhistory_commit
> + '[' '!' -d /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory
> ']'
> + cd /home/ol/setup-scripts/build/tmp-angstrom_2010_x-eglibc/buildhistory/
> + '[' '!' -d .git ']'
> ++ git status --porcelain
> + repostatus=
> + '[' '' '!=' '' ']'

We have been discussing this topic past couple days here:
http://thread.gmane.org/gmane.linux.distributions.angstrom.devel/6260/focus=6274

Moreover, bitbake-devel is probably the wrongest list for this discussion or 
patch. I'm copying other related lists, but probably the main one should be 
openembedded-core, as the patch tries to fix it's package_ipk class...


> From what I can tell, this is caused because the log_check command in
> setup-scripts/sources/openembedded-core/meta/classes/package_ipk.class
> matches on line 177 to
> 
> file://3.2.1/0042-xhci-Properly-handle-COMP_2ND_BW_ERR.patch
> 
> and to
> 
> file://3.2.6/0072-target-Fail-INQUIRY-commands-with-EVPD-0-but-PAGE-CO.patch

And your analysis seems correct, as the grep matches the patch names of 
beaglebone kernel from meta-ti.


> It occurs to me that this might be a result of the level of
> verbosity/debugging that I specified.

Actually, it does not depend on the verbosity/debug level, as it controls the 
output on the console and not in the log.do_rootfs file.


> Attached is my patch that does two things:
> 1) makes the error message from log_check slightly more informative

I don't think this part is really necessary - you always know which packaging 
system you use, so ipk/deb/rpm_log_check is rather easily identifiable...


> 2) changes the text that log_check is using to identify errors to
> incorporate regular expressions

So, this changes the regular expression to look for ^ERR and ^Fail at the 
beginning of the line...


> This makes it less likely to issue false positives, I hope that it
> doesn't increase the changes of false negatives, but I'm a rank
> amateur at this, so I wouldn't bet on it.

While the change should fix this particular issue, I'm also worried about 
increasing chances of missing the actual errors when they don't start with ERR 
and Fail at the beginning of the line.

Either way, you may want to resubmit the cleaned up patch to openembedded-core 
list for other to comment.

P.S. Sorry for somewhat excessive cross-posting.

-- 
Denys