[YP 2.1] [RFC PATCH] insane.bbclass: check files under /etc/ are marked by CONFFILES

[YP 2.1] [RFC PATCH] insane.bbclass: check files under /etc/ are marked by CONFFILES

[Robert Yang]

Hello,

Usually, files under /etc/ should be marked by CONFFILES so that package
manager can handle it well when package ugprades.

Please feel free to let me know if you have any suggestions, and I will
send patches later (including set CONFFILES for the recipes) if this is
OK.

[YOCTO #8413]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/insane.bbclass |   32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index fb04215..30fc41b 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -32,7 +32,7 @@ WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \
             installed-vs-shipped compile-host-path install-host-path \
             pn-overrides infodir build-deps file-rdeps \
             unknown-configure-option symlink-to-sysroot multilib \
-            invalid-pkgconfig host-user-contaminated \
+            invalid-pkgconfig host-user-contaminated conffiles \
             "
 ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
             perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
@@ -1020,6 +1020,36 @@ def package_qa_check_host_user(path, name, d, elf, messages):
             return False
     return True
 
+QAPATHTEST[conffiles] = "package_qa_check_conffiles"
+def package_qa_check_conffiles(path, name, d, elf, messages):
+    """ Check files in ${sysconfdir} are marked by CONFFILES """
+
+    if os.path.islink(path) or not os.path.exists(path):
+        return True
+
+    sysconfdir = d.getVar('sysconfdir', True)
+    dest = d.getVar('PKGDEST', True)
+    sysconfdir_path = os.path.join(os.path.join(dest, name), \
+                        sysconfdir.lstrip('/'))
+
+    if not path.startswith(sysconfdir_path):
+        return True
+    else:
+        conffiles = d.getVar('CONFFILES_%s' % name, True)
+        if conffiles == None:
+            conffiles = d.getVar('CONFFILES', True) or ""
+        conffiles = conffiles.split()
+        conf_orig_list = files_from_filevars(conffiles)
+        # PKGDEST + name
+        pkg_root = os.path.join(dest, name)
+        # Prepend "." to match items in files_from_filevars()
+        path_no_root = "." + path.replace(pkg_root, '')
+        if not conffiles or path_no_root not in conf_orig_list:
+            messages["conffiles"] = "%s: %s is not in CONFFILES" % (name, path)
+            return False
+
+    return True
+
 # The PACKAGE FUNC to scan each package
 python do_package_qa () {
     import subprocess
-- 
1.7.9.5

Re: [YP 2.1] [RFC PATCH] insane.bbclass: check files under /etc/ are marked by CONFFILES

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

On Fri, Oct 2, 2015 at 9:36 PM, Robert Yang <liezhi.yang@windriver.com>
wrote:

> +QAPATHTEST[conffiles] = "package_qa_check_conffiles"
> +def package_qa_check_conffiles(path, name, d, elf, messages):
> +    """ Check files in ${sysconfdir} are marked by CONFFILES """
> +
> +    if os.path.islink(path) or not os.path.exists(path):
> +        return True
> +
> +    sysconfdir = d.getVar('sysconfdir', True)
> +    dest = d.getVar('PKGDEST', True)
> +    sysconfdir_path = os.path.join(os.path.join(dest, name), \
> +                        sysconfdir.lstrip('/'))
> +
> +    if not path.startswith(sysconfdir_path):
> +        return True
>

This is missing a trailing slash, so e.g. checking /etcfoo would match, not
just /etc. Whenever you use startswith() on a path, remember to append '/'
or os.sep to the string being checked (of course, if we also need to match
just '/etc' without anything further, which I don't think is the case here,
you'd also have to check explicitly with ==, as '/etc/' won't be found at
the beginning of '/etc'. It's a common thing to miss, I think we have other
occurrences of this in the metadata :)
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1938 bytes --]

Re: [YP 2.1] [RFC PATCH] insane.bbclass: check files under /etc/ are marked by CONFFILES

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]

On Sat, Oct 3, 2015 at 11:39 AM, Christopher Larson <clarson@kergoth.com>
wrote:

> On Fri, Oct 2, 2015 at 9:36 PM, Robert Yang <liezhi.yang@windriver.com>
> wrote:
>
>> +QAPATHTEST[conffiles] = "package_qa_check_conffiles"
>> +def package_qa_check_conffiles(path, name, d, elf, messages):
>> +    """ Check files in ${sysconfdir} are marked by CONFFILES """
>> +
>> +    if os.path.islink(path) or not os.path.exists(path):
>> +        return True
>> +
>> +    sysconfdir = d.getVar('sysconfdir', True)
>> +    dest = d.getVar('PKGDEST', True)
>> +    sysconfdir_path = os.path.join(os.path.join(dest, name), \
>> +                        sysconfdir.lstrip('/'))
>> +
>> +    if not path.startswith(sysconfdir_path):
>> +        return True
>>
>
> This is missing a trailing slash, so e.g. checking /etcfoo would match,
> not just /etc. Whenever you use startswith() on a path, remember to append
> '/' or os.sep to the string being checked


To clarify, os.sep needs adding to sysconfdir_path, not path :) I realized
my wording was ambiguous.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2019 bytes --]

Re: [YP 2.1] [RFC PATCH] insane.bbclass: check files under /etc/ are marked by CONFFILES

[Robert Yang]

On 10/04/2015 02:40 AM, Christopher Larson wrote:
>
> On Sat, Oct 3, 2015 at 11:39 AM, Christopher Larson <clarson@kergoth.com
> <mailto:clarson@kergoth.com>> wrote:
>
>     On Fri, Oct 2, 2015 at 9:36 PM, Robert Yang <liezhi.yang@windriver.com
>     <mailto:liezhi.yang@windriver.com>> wrote:
>
>         +QAPATHTEST[conffiles] = "package_qa_check_conffiles"
>         +def package_qa_check_conffiles(path, name, d, elf, messages):
>         +    """ Check files in ${sysconfdir} are marked by CONFFILES """
>         +
>         +    if os.path.islink(path) or not os.path.exists(path):
>         +        return True
>         +
>         +    sysconfdir = d.getVar('sysconfdir', True)
>         +    dest = d.getVar('PKGDEST', True)
>         +    sysconfdir_path = os.path.join(os.path.join(dest, name), \
>         +                        sysconfdir.lstrip('/'))
>         +
>         +    if not path.startswith(sysconfdir_path):
>         +        return True
>
>
>     This is missing a trailing slash, so e.g. checking /etcfoo would match, not
>     just /etc. Whenever you use startswith() on a path, remember to append '/'
>     or os.sep to the string being checked
>
>
> To clarify, os.sep needs adding to sysconfdir_path, not path :) I realized my
> wording was ambiguous.

Thanks, after a world testing, there 187 binary rpm packages have this issue
(have /etc/foo, but no CONFFILES set), how about we do this as Debian:
1) Treat all the files under /etc/ as CONFFILES
2) Move the files out or /etc/ if it is not a conffile
3) If the file /etc/foo is a conffile, and is changed during runtime,
    then create a symlink /etc/foo -> /var/foo

// Robert

> --
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics