[PATCH 0/2] warn when ALTERNATIVE_PRIORITY are the same

[PATCH 0/2] warn when ALTERNATIVE_PRIORITY are the same

[Chen Qi]

The following changes since commit 5e21afc9395060b489156d3f90505a372b713f37:

  Revert "selftest/wic: extending test coverage for WIC script options" (2016-12-20 17:06:01 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib ChenQi/BUG8314
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/BUG8314

Chen Qi (2):
  package_manager: default to have scriptlet output captured in log
  opkg-utils: warn if update-alternatives finds priority conflict

 meta/lib/oe/package_manager.py                     |  9 ++++----
 ...rnatives-warn-when-multiple-providers-hav.patch | 26 ++++++++++++++++++++++
 meta/recipes-devtools/opkg-utils/opkg-utils_git.bb |  3 ++-
 3 files changed, 33 insertions(+), 5 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-update-alternatives-warn-when-multiple-providers-hav.patch

-- 
1.9.1

[PATCH 1/2] package_manager: default to have scriptlet output captured in log

[Chen Qi]

We need to have scriptlet output captured in log. If we don't do so,
some useful information from scriptlets (especially postinstall script)
would be missing. In case a script has a warning message but it does not
necessarily have to fail, the message should be captured.

Opkg has already done that. Change for rpm and dpkg so that scriptlet
output is captured and no warning message is missing.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/lib/oe/package_manager.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index e557473..6892106 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -669,11 +669,11 @@ class RpmPM(PackageManager):
         self.install_dir_path = os.path.join(self.target_rootfs, self.install_dir_name)
         self.rpm_cmd = bb.utils.which(os.getenv('PATH'), "rpm")
         self.smart_cmd = bb.utils.which(os.getenv('PATH'), "smart")
-        # 0 = default, only warnings
-        # 1 = --log-level=info (includes information about executing scriptlets and their output)
+        # 0 = --log-level=warning, only warnings
+        # 1 = --log-level=info (includes information about executing scriptlets and their output), default
         # 2 = --log-level=debug
         # 3 = --log-level=debug plus dumps of scriplet content and command invocation
-        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "0")
+        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "1")
         self.smart_opt = ["--log-level=%s" %
                          ("warning" if self.debug_level == 0 else
                           "info" if self.debug_level == 1 else
@@ -2025,7 +2025,8 @@ class DpkgPM(OpkgDpkgPM):
                         bb.note("Executing %s for package: %s ..." %
                                  (control_script.name.lower(), pkg_name))
                         subprocess.check_output([p_full, control_script.argument],
-                                stderr=subprocess.STDOUT)
+                                stderr=subprocess.STDOUT).decode("utf-8")
+                        bb.note(output)
                     except subprocess.CalledProcessError as e:
                         bb.note("%s for package %s failed with %d:\n%s" %
                                 (control_script.name, pkg_name, e.returncode,
-- 
1.9.1

Re: [PATCH 1/2] package_manager: default to have scriptlet output captured in log

[Jussi Kukkonen]

[-- Attachment #1: Type: text/plain, Size: 6348 bytes --]

On 21 December 2016 at 06:32, Chen Qi <Qi.Chen@windriver.com> wrote:
>
> We need to have scriptlet output captured in log. If we don't do so,
> some useful information from scriptlets (especially postinstall script)
> would be missing. In case a script has a warning message but it does not
> necessarily have to fail, the message should be captured.
>
> Opkg has already done that. Change for rpm and dpkg so that scriptlet
> output is captured and no warning message is missing.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/lib/oe/package_manager.py | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/meta/lib/oe/package_manager.py
b/meta/lib/oe/package_manager.py
> index e557473..6892106 100644
> --- a/meta/lib/oe/package_manager.py
> +++ b/meta/lib/oe/package_manager.py
> @@ -669,11 +669,11 @@ class RpmPM(PackageManager):
>          self.install_dir_path = os.path.join(self.target_rootfs,
self.install_dir_name)
>          self.rpm_cmd = bb.utils.which(os.getenv('PATH'), "rpm")
>          self.smart_cmd = bb.utils.which(os.getenv('PATH'), "smart")
> -        # 0 = default, only warnings
> -        # 1 = --log-level=info (includes information about executing
scriptlets and their output)
> +        # 0 = --log-level=warning, only warnings
> +        # 1 = --log-level=info (includes information about executing
scriptlets and their output), default
>          # 2 = --log-level=debug
>          # 3 = --log-level=debug plus dumps of scriplet content and
command invocation
> -        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "0")
> +        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "1")
>          self.smart_opt = ["--log-level=%s" %
>                           ("warning" if self.debug_level == 0 else
>                            "info" if self.debug_level == 1 else
> @@ -2025,7 +2025,8 @@ class DpkgPM(OpkgDpkgPM):
>                          bb.note("Executing %s for package: %s ..." %
>                                   (control_script.name.lower(), pkg_name))
>                          subprocess.check_output([p_full,
control_script.argument],
> -                                stderr=subprocess.STDOUT)
> +                                stderr=subprocess.STDOUT).decode("utf-8")
> +                        bb.note(output)


This patch seems likely to be the reason for this autobuilder failure:
https://autobuilder.yoctoproject.org/main/builders/nightly-deb-non-deb/builds/669/steps/BuildImages/logs/stdio

---
ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Error executing a python
function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_rootfs(d)
     0003:
File:
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/classes/image.bbclass',
lineno: 266, function: do_rootfs
     0262:
     0263:    progress_reporter.next_stage()
     0264:
     0265:    # generate rootfs
 *** 0266:    create_rootfs(d, progress_reporter=progress_reporter,
logcatcher=logcatcher)
     0267:
     0268:    progress_reporter.finish()
     0269:}
     0270:do_rootfs[dirs] = "${TOPDIR}"
File:
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py',
lineno: 1009, function: create_rootfs
     1005:        RpmRootfs(d, manifest_dir, progress_reporter,
logcatcher).create()
     1006:    elif img_type == "ipk":
     1007:        OpkgRootfs(d, manifest_dir, progress_reporter,
logcatcher).create()
     1008:    elif img_type == "deb":
 *** 1009:        DpkgRootfs(d, manifest_dir, progress_reporter,
logcatcher).create()
     1010:
     1011:    os.environ.clear()
     1012:    os.environ.update(env_bkp)
     1013:
File:
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py',
lineno: 202, function: create
     0198:        if self.progress_reporter:
     0199:            self.progress_reporter.next_stage()
     0200:
     0201:        # call the package manager dependent create method
 *** 0202:        self._create()
     0203:
     0204:        sysconfdir = self.image_rootfs +
self.d.getVar('sysconfdir')
     0205:        bb.utils.mkdirhier(sysconfdir)
     0206:        with open(sysconfdir + "/version", "w+") as ver:
File:
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py',
lineno: 691, function: _create
     0687:        self.pm.fix_broken_dependencies()
     0688:
     0689:        self.pm.mark_packages("installed")
     0690:
 *** 0691:        self.pm.run_pre_post_installs()
     0692:
     0693:        execute_pre_post_process(self.d, deb_post_process_cmds)
     0694:
     0695:        if self.progress_reporter:
File:
'/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/package_manager.py',
lineno: 2029, function: run_pre_post_installs
     2025:                        bb.note("Executing %s for package: %s
..." %
     2026:                                 (control_script.name.lower(),
pkg_name))
     2027:                        subprocess.check_output([p_full,
control_script.argument],
     2028:
 stderr=subprocess.STDOUT).decode("utf-8")
 *** 2029:                        bb.note(output)
     2030:                    except subprocess.CalledProcessError as e:
     2031:                        bb.note("%s for package %s failed with
%d:\n%s" %
     2032:                                (control_script.name, pkg_name,
e.returncode,
     2033:                                    e.output.decode("utf-8")))
Exception: NameError: name 'output' is not defined
---


Thanks,
  Jussi


>
>                      except subprocess.CalledProcessError as e:
>                          bb.note("%s for package %s failed with %d:\n%s" %
>                                  (control_script.name, pkg_name,
e.returncode,
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[-- Attachment #2: Type: text/html, Size: 8101 bytes --]

Re: [PATCH 1/2] package_manager: default to have scriptlet output captured in log

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 6831 bytes --]

On 12/22/2016 05:36 PM, Jussi Kukkonen wrote:
> On 21 December 2016 at 06:32, Chen Qi <Qi.Chen@windriver.com 
> <mailto:Qi.Chen@windriver.com>> wrote:
> >
> > We need to have scriptlet output captured in log. If we don't do so,
> > some useful information from scriptlets (especially postinstall script)
> > would be missing. In case a script has a warning message but it does not
> > necessarily have to fail, the message should be captured.
> >
> > Opkg has already done that. Change for rpm and dpkg so that scriptlet
> > output is captured and no warning message is missing.
> >
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com 
> <mailto:Qi.Chen@windriver.com>>
> > ---
> >  meta/lib/oe/package_manager.py | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/lib/oe/package_manager.py 
> b/meta/lib/oe/package_manager.py
> > index e557473..6892106 100644
> > --- a/meta/lib/oe/package_manager.py
> > +++ b/meta/lib/oe/package_manager.py
> > @@ -669,11 +669,11 @@ class RpmPM(PackageManager):
> >          self.install_dir_path = os.path.join(self.target_rootfs, 
> self.install_dir_name)
> >          self.rpm_cmd = bb.utils.which(os.getenv('PATH'), "rpm")
> >          self.smart_cmd = bb.utils.which(os.getenv('PATH'), "smart")
> > -        # 0 = default, only warnings
> > -        # 1 = --log-level=info (includes information about 
> executing scriptlets and their output)
> > +        # 0 = --log-level=warning, only warnings
> > +        # 1 = --log-level=info (includes information about 
> executing scriptlets and their output), default
> >          # 2 = --log-level=debug
> >          # 3 = --log-level=debug plus dumps of scriplet content and 
> command invocation
> > -        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "0")
> > +        self.debug_level = int(d.getVar('ROOTFS_RPM_DEBUG') or "1")
> >          self.smart_opt = ["--log-level=%s" %
> >                           ("warning" if self.debug_level == 0 else
> >                            "info" if self.debug_level == 1 else
> > @@ -2025,7 +2025,8 @@ class DpkgPM(OpkgDpkgPM):
> >                          bb.note("Executing %s for package: %s ..." %
> > (control_script.name.lower(), pkg_name))
> >                          subprocess.check_output([p_full, 
> control_script.argument],
> > -                                stderr=subprocess.STDOUT)
> > +  stderr=subprocess.STDOUT).decode("utf-8")
> > +                        bb.note(output)
>
>
> This patch seems likely to be the reason for this autobuilder failure:
> https://autobuilder.yoctoproject.org/main/builders/nightly-deb-non-deb/builds/669/steps/BuildImages/logs/stdio
>

Sorry, I'll fix it immediately and send out V2.

Best Regards,
Chen Qi

> ---
> ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Error executing a python 
> function in exec_python_func() autogenerated:
>
> The stack trace of python calls that resulted in this 
> exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:do_rootfs(d)
>      0003:
> File: 
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/classes/image.bbclass', 
> lineno: 266, function: do_rootfs
>      0262:
>      0263:    progress_reporter.next_stage()
>      0264:
>      0265:    # generate rootfs
>  *** 0266:    create_rootfs(d, progress_reporter=progress_reporter, 
> logcatcher=logcatcher)
>      0267:
>      0268:    progress_reporter.finish()
>      0269:}
>      0270:do_rootfs[dirs] = "${TOPDIR}"
> File: 
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py', 
> lineno: 1009, function: create_rootfs
>      1005:        RpmRootfs(d, manifest_dir, progress_reporter, 
> logcatcher).create()
>      1006:    elif img_type == "ipk":
>      1007:        OpkgRootfs(d, manifest_dir, progress_reporter, 
> logcatcher).create()
>      1008:    elif img_type == "deb":
>  *** 1009:        DpkgRootfs(d, manifest_dir, progress_reporter, 
> logcatcher).create()
>      1010:
>      1011:    os.environ.clear()
>      1012:    os.environ.update(env_bkp)
>      1013:
> File: 
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py', 
> lineno: 202, function: create
>      0198:        if self.progress_reporter:
>      0199:            self.progress_reporter.next_stage()
>      0200:
>      0201:        # call the package manager dependent create method
>  *** 0202:        self._create()
>      0203:
>      0204:        sysconfdir = self.image_rootfs + 
> self.d.getVar('sysconfdir')
>      0205:        bb.utils.mkdirhier(sysconfdir)
>      0206:        with open(sysconfdir + "/version", "w+") as ver:
> File: 
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/rootfs.py', 
> lineno: 691, function: _create
>      0687:        self.pm.fix_broken_dependencies()
>      0688:
>      0689:        self.pm.mark_packages("installed")
>      0690:
>  *** 0691:        self.pm.run_pre_post_installs()
>      0692:
>      0693:        execute_pre_post_process(self.d, deb_post_process_cmds)
>      0694:
>      0695:        if self.progress_reporter:
> File: 
> '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-deb-non-deb/build/meta/lib/oe/package_manager.py', 
> lineno: 2029, function: run_pre_post_installs
>      2025:                        bb.note("Executing %s for package: 
> %s ..." %
>      2026: (control_script.name.lower(), pkg_name))
>      2027:  subprocess.check_output([p_full, control_script.argument],
>      2028:  stderr=subprocess.STDOUT).decode("utf-8")
>  *** 2029:                        bb.note(output)
>      2030:                    except subprocess.CalledProcessError as e:
>      2031:                        bb.note("%s for package %s failed 
> with %d:\n%s" %
>      2032:                                (control_script.name 
> <http://control_script.name>, pkg_name, e.returncode,
>      2033:  e.output.decode("utf-8")))
> Exception: NameError: name 'output' is not defined
> ---
>
>
> Thanks,
>   Jussi
>
>
> >
> >                      except subprocess.CalledProcessError as e:
> >                          bb.note("%s for package %s failed with 
> %d:\n%s" %
> >                                  (control_script.name 
> <http://control_script.name>, pkg_name, e.returncode,
> > --
> > 1.9.1
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org 
> <mailto:Openembedded-core@lists.openembedded.org>
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core



[-- Attachment #2: Type: text/html, Size: 10550 bytes --]

[PATCH 2/2] opkg-utils: warn if update-alternatives finds priority conflict

[Chen Qi]

If multiple providers for a utility have the same alternatives priority,
which one would be chosen is determined by which one is installed later.
Our alternatives system should be able to detect such problem and warn users
so that potential problems could be avoided.

Modify update-alternatives to warn users when detecting multiple providers
with the same priority.

[YOCTO #8314]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 ...rnatives-warn-when-multiple-providers-hav.patch | 26 ++++++++++++++++++++++
 meta/recipes-devtools/opkg-utils/opkg-utils_git.bb |  3 ++-
 2 files changed, 28 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-update-alternatives-warn-when-multiple-providers-hav.patch

diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0001-update-alternatives-warn-when-multiple-providers-hav.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0001-update-alternatives-warn-when-multiple-providers-hav.patch
new file mode 100644
index 0000000..afce1e1
--- /dev/null
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0001-update-alternatives-warn-when-multiple-providers-hav.patch
@@ -0,0 +1,26 @@
+Subject: update-alternatives: warn when multiple providers have the same priority
+
+Upstream-Status: Pending
+
+Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
+---
+ update-alternatives | 3 +++
+ 1 file changed, 3 insertions(+)
+
+diff --git a/update-alternatives b/update-alternatives
+index ca01d5d..ffad853 100644
+--- a/update-alternatives
++++ b/update-alternatives
+@@ -90,6 +90,9 @@ add_alt() {
+ 	local path="$2"
+ 	local priority="$3"
+ 	remove_alt $name $path
++	if grep -qw "$priority" $ad/$name; then
++		echo "Warn: update-alternatives: $name has multiple providers with the same priority, please check $ad/$name for details"
++	fi
+ 	echo "$path $priority" >> $ad/$name
+ }
+ 
+-- 
+2.8.3
+
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
index 19a852e..7b01bfc 100644
--- a/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
+++ b/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb
@@ -10,7 +10,8 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu
 SRCREV = "3ffece9bf19a844edacc563aa092fd1fbfcffeee"
 PV = "0.3.2+git${SRCPV}"
 
-SRC_URI = "git://git.yoctoproject.org/opkg-utils"
+SRC_URI = "git://git.yoctoproject.org/opkg-utils \
+           file://0001-update-alternatives-warn-when-multiple-providers-hav.patch"
 SRC_URI_append_class-native = " file://tar_ignore_error.patch"
 
 S = "${WORKDIR}/git"
-- 
1.9.1