* [PATCH v2 0/2] Create systemd users at build time
@ 2016-05-09 23:42 Stephano Cetola
2016-05-09 23:42 ` [PATCH v2 1/2] systemd: add sysusers config option to defaults Stephano Cetola
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Stephano Cetola @ 2016-05-09 23:42 UTC (permalink / raw)
To: openembedded-core
Systemd has a config option sysusers which creates any users or groups that
systemd needs on first boot. There is a rootfs-postcommand that creates these
users in case the filesystem is readonly, however it does not take into
account any staicids that the user may have set.
By adding these users at build time, we allow a check for staticids, which would
otherwise have been skipped.
This issue was initially raised by a fix that was added to Ostro OS. See
Patrick's comments for more info on that.
[ YOCTO #9497 ]
Changed since v1:
* added oe-core prefix to subject line
Stephano Cetola (2):
systemd: add sysusers config option to defaults
systemd: add users/groups at build rather than runtime
meta/classes/rootfs-postcommands.bbclass | 66 +++++++++++++++++++++-----------
meta/classes/useradd-staticids.bbclass | 56 +--------------------------
meta/recipes-core/systemd/systemd_229.bb | 4 +-
useradd-staticids-list.bbclass | 56 +++++++++++++++++++++++++++
4 files changed, 103 insertions(+), 79 deletions(-)
create mode 100644 useradd-staticids-list.bbclass
--
2.8.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH v2 1/2] systemd: add sysusers config option to defaults 2016-05-09 23:42 [PATCH v2 0/2] Create systemd users at build time Stephano Cetola @ 2016-05-09 23:42 ` Stephano Cetola 2016-05-09 23:50 ` Khem Raj 2016-05-09 23:42 ` [PATCH v2 2/2] systemd: add users/groups at build rather than runtime Stephano Cetola 2016-05-11 10:46 ` [PATCH v2 0/2] Create systemd users at build time Puustinen, Ismo 2 siblings, 1 reply; 12+ messages in thread From: Stephano Cetola @ 2016-05-09 23:42 UTC (permalink / raw) To: openembedded-core Without the sysusers config option, the build will not know which users need to be added for systemd. This is especially necessary if the filesystem is readonly, as these users cannot be added at runtime. Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> --- meta/recipes-core/systemd/systemd_229.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/meta/recipes-core/systemd/systemd_229.bb b/meta/recipes-core/systemd/systemd_229.bb index c23c749..aba73df 100644 --- a/meta/recipes-core/systemd/systemd_229.bb +++ b/meta/recipes-core/systemd/systemd_229.bb @@ -23,6 +23,7 @@ DEPENDS = "kmod docbook-sgml-dtd-4.1-native intltool-native gperf-native acl rea SECTION = "base/shell" inherit useradd pkgconfig autotools perlnative update-rc.d update-alternatives qemu systemd ptest gettext bash-completion +USERADDEXTENSION += " useradd-staticids" SRCREV = "714c62b46379abb7558c544665522aca91691e10" @@ -80,7 +81,8 @@ PACKAGECONFIG ??= "xz \ quotacheck \ bootchart \ hostnamed \ - ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname sysusers', '', d)} \ + sysusers \ + ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname', '', d)} \ hibernate \ timedated \ timesyncd \ -- 2.8.0 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2 1/2] systemd: add sysusers config option to defaults 2016-05-09 23:42 ` [PATCH v2 1/2] systemd: add sysusers config option to defaults Stephano Cetola @ 2016-05-09 23:50 ` Khem Raj 2016-05-10 0:24 ` Stephano Cetola 0 siblings, 1 reply; 12+ messages in thread From: Khem Raj @ 2016-05-09 23:50 UTC (permalink / raw) To: Stephano Cetola; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 1863 bytes --] > On May 9, 2016, at 4:42 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: > > Without the sysusers config option, the build will not know which users > need to be added for systemd. This is especially necessary if the > filesystem is readonly, as these users cannot be added at runtime. > > Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> > --- > meta/recipes-core/systemd/systemd_229.bb | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/meta/recipes-core/systemd/systemd_229.bb b/meta/recipes-core/systemd/systemd_229.bb > index c23c749..aba73df 100644 > --- a/meta/recipes-core/systemd/systemd_229.bb > +++ b/meta/recipes-core/systemd/systemd_229.bb > @@ -23,6 +23,7 @@ DEPENDS = "kmod docbook-sgml-dtd-4.1-native intltool-native gperf-native acl rea > SECTION = "base/shell" > > inherit useradd pkgconfig autotools perlnative update-rc.d update-alternatives qemu systemd ptest gettext bash-completion > +USERADDEXTENSION += " useradd-staticids" > > SRCREV = "714c62b46379abb7558c544665522aca91691e10" > > @@ -80,7 +81,8 @@ PACKAGECONFIG ??= "xz \ > quotacheck \ > bootchart \ > hostnamed \ > - ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname sysusers', '', d)} \ > + sysusers \ > + ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname', '', d)} \ sysusers did not compile with uclibc, did you test this patch with uclibc? > hibernate \ > timedated \ > timesyncd \ > -- > 2.8.0 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 211 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 1/2] systemd: add sysusers config option to defaults 2016-05-09 23:50 ` Khem Raj @ 2016-05-10 0:24 ` Stephano Cetola 0 siblings, 0 replies; 12+ messages in thread From: Stephano Cetola @ 2016-05-10 0:24 UTC (permalink / raw) To: Khem Raj; +Cc: openembedded-core On 05/09, Khem Raj wrote: > > > On May 9, 2016, at 4:42 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: > > > > Without the sysusers config option, the build will not know which users > > need to be added for systemd. This is especially necessary if the > > filesystem is readonly, as these users cannot be added at runtime. > > > > Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> > > --- > > meta/recipes-core/systemd/systemd_229.bb | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/meta/recipes-core/systemd/systemd_229.bb b/meta/recipes-core/systemd/systemd_229.bb > > index c23c749..aba73df 100644 > > --- a/meta/recipes-core/systemd/systemd_229.bb > > +++ b/meta/recipes-core/systemd/systemd_229.bb > > @@ -23,6 +23,7 @@ DEPENDS = "kmod docbook-sgml-dtd-4.1-native intltool-native gperf-native acl rea > > SECTION = "base/shell" > > > > inherit useradd pkgconfig autotools perlnative update-rc.d update-alternatives qemu systemd ptest gettext bash-completion > > +USERADDEXTENSION += " useradd-staticids" > > > > SRCREV = "714c62b46379abb7558c544665522aca91691e10" > > > > @@ -80,7 +81,8 @@ PACKAGECONFIG ??= "xz \ > > quotacheck \ > > bootchart \ > > hostnamed \ > > - ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname sysusers', '', d)} \ > > + sysusers \ > > + ${@bb.utils.contains('TCLIBC', 'glibc', 'myhostname', '', d)} \ > > sysusers did not compile with uclibc, did you test this patch with uclibc? You are correct, it still fails. Sorry, you can drop this from the series. > > > hibernate \ > > timedated \ > > timesyncd \ > > -- > > 2.8.0 > > > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 2/2] systemd: add users/groups at build rather than runtime 2016-05-09 23:42 [PATCH v2 0/2] Create systemd users at build time Stephano Cetola 2016-05-09 23:42 ` [PATCH v2 1/2] systemd: add sysusers config option to defaults Stephano Cetola @ 2016-05-09 23:42 ` Stephano Cetola 2016-05-09 23:49 ` Khem Raj 2016-05-11 10:46 ` [PATCH v2 0/2] Create systemd users at build time Puustinen, Ismo 2 siblings, 1 reply; 12+ messages in thread From: Stephano Cetola @ 2016-05-09 23:42 UTC (permalink / raw) To: openembedded-core This fix ensures that systemd users are created at build and that any staticids for the users or groups are preserved. This functionality only existed for readonly filesystems, however it seems wise to be sure these users are added rather than relying on systemd to create them. [ YOCTO #9497 ] Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> --- meta/classes/rootfs-postcommands.bbclass | 66 +++++++++++++++++++++----------- meta/classes/useradd-staticids.bbclass | 56 +-------------------------- useradd-staticids-list.bbclass | 56 +++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 78 deletions(-) create mode 100644 useradd-staticids-list.bbclass diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass index 95d28af..db4cc75 100644 --- a/meta/classes/rootfs-postcommands.bbclass +++ b/meta/classes/rootfs-postcommands.bbclass @@ -1,4 +1,4 @@ - +inherit useradd-staticids-list # Zap the root password if debug-tweaks feature is not enabled ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'debug-tweaks', 'empty-root-password' ], "", "zap_empty_root_password ; ",d)}' @@ -23,6 +23,9 @@ POSTINST_LOGFILE ?= "${localstatedir}/log/postinstall.log" SYSTEMD_DEFAULT_TARGET ?= '${@bb.utils.contains("IMAGE_FEATURES", "x11-base", "graphical.target", "multi-user.target", d)}' ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "set_systemd_default_target; ", "", d)}' +# Create systemd users at build rather than at runtime +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "create_systemd_sysusers; ", "", d)}' + ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' # Disable DNS lookups, the SSH_DISABLE_DNS_LOOKUP can be overridden to allow @@ -30,7 +33,45 @@ ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' SSH_DISABLE_DNS_LOOKUP ?= " ssh_disable_dns_lookup ; " ROOTFS_POSTPROCESS_COMMAND_append_qemuall = "${SSH_DISABLE_DNS_LOOKUP}" - +# Add systemd users checking for any staticids +python create_systemd_sysusers () { + import shlex + import subprocess + s_users = merge_files(get_passwd_list(d), 7) + s_groups = merge_files(get_group_list(d), 4) + confs = ['%s/usr/lib/sysusers.d/systemd.conf' % d.getVar('IMAGE_ROOTFS', d), '%s/usr/lib/sysusers.d/systemd-remote.conf' % d.getVar('IMAGE_ROOTFS', d)] + for conf in confs: + if os.path.isfile(conf): + with open(conf, 'r') as f: + for line in f: + if line.startswith('#'): + continue + fields = shlex.split(line.rstrip()) + if not fields: + continue + if fields[0] == 'u': + useradd_params = '' + if fields[1] in s_users: + useradd_params += '--uid %s' % s_users[fields[1]][2] + bb.warn("Changing user %s's uid to (%s), verify configuration files!" % (fields[1], s_users[fields[1]][2])) + elif fields[2] != '-': + useradd_params += '--uid %s' % fields[2] + if fields[3] != '-': + useradd_params += ' --comment "%s"' % fields[3] + useradd_params += ' --root "%s" --shell /sbin/nologin --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) + code = subprocess.call('useradd %s' % useradd_params, shell=True) + elif fields[0] == 'g': + groupadd_params = "" + if fields[1] in s_groups: + groupadd_params += '--gid %s' % s_groups[fields[1]][2] + bb.warn("Changing group %s's gid to (%s), verify configuration files!" % (fields[1], s_groups[fields[1]][2])) + elif fields[2] != '-': + groupadd_params += '--gid %s' % fields[2] + if fields[3] != '-': + groupadd_params += ' --comment "%s"' % (d.getVar("IMAGE_ROOTFS", d), fields[3]) + groupadd_params += ' --root "%s" --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) + code = subprocess.call('groupadd %s' % groupadd_params, shell=True) +} # # A hook function to support read-only-rootfs IMAGE_FEATURES @@ -73,27 +114,6 @@ read_only_rootfs_hook () { ${IMAGE_ROOTFS}/etc/init.d/populate-volatile.sh fi fi - - if ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "true", "false", d)}; then - # Update user database files so that services don't fail for a read-only systemd system - for conffile in ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd.conf ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd-remote.conf; do - [ -e $conffile ] || continue - grep -v "^#" $conffile | sed -e '/^$/d' | while read type name id comment; do - if [ "$type" = "u" ]; then - useradd_params="" - [ "$id" != "-" ] && useradd_params="$useradd_params --uid $id" - [ "$comment" != "-" ] && useradd_params="$useradd_params --comment $comment" - useradd_params="$useradd_params --system $name" - eval useradd --root ${IMAGE_ROOTFS} $useradd_params || true - elif [ "$type" = "g" ]; then - groupadd_params="" - [ "$id" != "-" ] && groupadd_params="$groupadd_params --gid $id" - groupadd_params="$groupadd_params --system $name" - eval groupadd --root ${IMAGE_ROOTFS} $groupadd_params || true - fi - done - done - fi } # diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass index a9b506d..e9dfe69 100644 --- a/meta/classes/useradd-staticids.bbclass +++ b/meta/classes/useradd-staticids.bbclass @@ -1,3 +1,4 @@ +inherit useradd-staticids-list # In order to support a deterministic set of 'dynamic' users/groups, # we need a function to reformat the params based on a static file def update_useradd_static_config(d): @@ -17,35 +18,6 @@ def update_useradd_static_config(d): def error(self, message): raise bb.build.FuncFailed(message) - def list_extend(iterable, length, obj = None): - """Ensure that iterable is the specified length by extending with obj - and return it as a list""" - return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) - - def merge_files(file_list, exp_fields): - """Read each passwd/group file in file_list, split each line and create - a dictionary with the user/group names as keys and the split lines as - values. If the user/group name already exists in the dictionary, then - update any fields in the list with the values from the new list (if they - are set).""" - id_table = dict() - for conf in file_list.split(): - if os.path.exists(conf): - f = open(conf, "r") - for line in f: - if line.startswith('#'): - continue - # Make sure there always are at least exp_fields elements in - # the field list. This allows for leaving out trailing - # colons in the files. - fields = list_extend(line.rstrip().split(":"), exp_fields) - if fields[0] not in id_table: - id_table[fields[0]] = fields - else: - id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) - - return id_table - # We parse and rewrite the useradd components def rewrite_useradd(params): # The following comes from --help on useradd from shadow @@ -73,19 +45,6 @@ def update_useradd_static_config(d): parser.add_argument("-U", "--user-group", help="create a group with the same name as the user", action="store_const", const=True) parser.add_argument("LOGIN", help="Login name of the new user") - # Return a list of configuration files based on either the default - # files/passwd or the contents of USERADD_UID_TABLES - # paths are resolved via BBPATH - def get_passwd_list(d): - str = "" - bbpath = d.getVar('BBPATH', True) - passwd_tables = d.getVar('USERADD_UID_TABLES', True) - if not passwd_tables: - passwd_tables = 'files/passwd' - for conf_file in passwd_tables.split(): - str += " %s" % bb.utils.which(bbpath, conf_file) - return str - newparams = [] users = None for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): @@ -206,19 +165,6 @@ def update_useradd_static_config(d): parser.add_argument("-r", "--system", help="create a system account", action="store_true") parser.add_argument("GROUP", help="Group name of the new group") - # Return a list of configuration files based on either the default - # files/group or the contents of USERADD_GID_TABLES - # paths are resolved via BBPATH - def get_group_list(d): - str = "" - bbpath = d.getVar('BBPATH', True) - group_tables = d.getVar('USERADD_GID_TABLES', True) - if not group_tables: - group_tables = 'files/group' - for conf_file in group_tables.split(): - str += " %s" % bb.utils.which(bbpath, conf_file) - return str - newparams = [] groups = None for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): diff --git a/useradd-staticids-list.bbclass b/useradd-staticids-list.bbclass new file mode 100644 index 0000000..377f748 --- /dev/null +++ b/useradd-staticids-list.bbclass @@ -0,0 +1,56 @@ +# Return a list of configuration files based on either the default +# files/group or the contents of USERADD_GID_TABLES +# paths are resolved via BBPATH +def get_group_list(d): + str = "" + bbpath = d.getVar('BBPATH', True) + group_tables = d.getVar('USERADD_GID_TABLES', True) + if not group_tables: + group_tables = 'files/group' + for conf_file in group_tables.split(): + str += " %s" % bb.utils.which(bbpath, conf_file) + return str + +def list_extend(iterable, length, obj = None): + import itertools + """Ensure that iterable is the specified length by extending with obj + and return it as a list""" + return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) + +def merge_files(file_list, exp_fields): + import itertools + """Read each passwd/group file in file_list, split each line and create + a dictionary with the user/group names as keys and the split lines as + values. If the user/group name already exists in the dictionary, then + update any fields in the list with the values from the new list (if they + are set).""" + id_table = dict() + for conf in file_list.split(): + if os.path.exists(conf): + f = open(conf, "r") + for line in f: + if line.startswith('#'): + continue + # Make sure there always are at least exp_fields elements in + # the field list. This allows for leaving out trailing + # colons in the files. + fields = list_extend(line.rstrip().split(":"), exp_fields) + if fields[0] not in id_table: + id_table[fields[0]] = fields + else: + id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) + + return id_table + +# Return a list of configuration files based on either the default +# files/passwd or the contents of USERADD_UID_TABLES +# paths are resolved via BBPATH +def get_passwd_list(d): + str = "" + bbpath = d.getVar('BBPATH', True) + passwd_tables = d.getVar('USERADD_UID_TABLES', True) + if not passwd_tables: + passwd_tables = 'files/passwd' + for conf_file in passwd_tables.split(): + str += " %s" % bb.utils.which(bbpath, conf_file) + return str -- 2.8.0 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2 2/2] systemd: add users/groups at build rather than runtime 2016-05-09 23:42 ` [PATCH v2 2/2] systemd: add users/groups at build rather than runtime Stephano Cetola @ 2016-05-09 23:49 ` Khem Raj 2016-05-10 0:55 ` Stephano Cetola 0 siblings, 1 reply; 12+ messages in thread From: Khem Raj @ 2016-05-09 23:49 UTC (permalink / raw) To: Stephano Cetola; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 13564 bytes --] > On May 9, 2016, at 4:42 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: > > This fix ensures that systemd users are created at build and that any > staticids for the users or groups are preserved. This functionality only > existed for readonly filesystems, however it seems wise to be sure these > users are added rather than relying on systemd to create them. We need to cover the case where online package management is used. If you were to fix it in systemd recipe itself which takes care of creating these users in post installs that will fix it for this case too. > > [ YOCTO #9497 ] > > Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> > --- > meta/classes/rootfs-postcommands.bbclass | 66 +++++++++++++++++++++----------- > meta/classes/useradd-staticids.bbclass | 56 +-------------------------- > useradd-staticids-list.bbclass | 56 +++++++++++++++++++++++++++ > 3 files changed, 100 insertions(+), 78 deletions(-) > create mode 100644 useradd-staticids-list.bbclass > > diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass > index 95d28af..db4cc75 100644 > --- a/meta/classes/rootfs-postcommands.bbclass > +++ b/meta/classes/rootfs-postcommands.bbclass > @@ -1,4 +1,4 @@ > - > +inherit useradd-staticids-list > # Zap the root password if debug-tweaks feature is not enabled > ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'debug-tweaks', 'empty-root-password' ], "", "zap_empty_root_password ; ",d)}' > > @@ -23,6 +23,9 @@ POSTINST_LOGFILE ?= "${localstatedir}/log/postinstall.log" > SYSTEMD_DEFAULT_TARGET ?= '${@bb.utils.contains("IMAGE_FEATURES", "x11-base", "graphical.target", "multi-user.target", d)}' > ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "set_systemd_default_target; ", "", d)}' > > +# Create systemd users at build rather than at runtime > +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "create_systemd_sysusers; ", "", d)}' > + > ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' > > # Disable DNS lookups, the SSH_DISABLE_DNS_LOOKUP can be overridden to allow > @@ -30,7 +33,45 @@ ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' > SSH_DISABLE_DNS_LOOKUP ?= " ssh_disable_dns_lookup ; " > ROOTFS_POSTPROCESS_COMMAND_append_qemuall = "${SSH_DISABLE_DNS_LOOKUP}" > > - > +# Add systemd users checking for any staticids > +python create_systemd_sysusers () { > + import shlex > + import subprocess > + s_users = merge_files(get_passwd_list(d), 7) > + s_groups = merge_files(get_group_list(d), 4) > + confs = ['%s/usr/lib/sysusers.d/systemd.conf' % d.getVar('IMAGE_ROOTFS', d), '%s/usr/lib/sysusers.d/systemd-remote.conf' % d.getVar('IMAGE_ROOTFS', d)] > + for conf in confs: > + if os.path.isfile(conf): > + with open(conf, 'r') as f: > + for line in f: > + if line.startswith('#'): > + continue > + fields = shlex.split(line.rstrip()) > + if not fields: > + continue > + if fields[0] == 'u': > + useradd_params = '' > + if fields[1] in s_users: > + useradd_params += '--uid %s' % s_users[fields[1]][2] > + bb.warn("Changing user %s's uid to (%s), verify configuration files!" % (fields[1], s_users[fields[1]][2])) > + elif fields[2] != '-': > + useradd_params += '--uid %s' % fields[2] > + if fields[3] != '-': > + useradd_params += ' --comment "%s"' % fields[3] > + useradd_params += ' --root "%s" --shell /sbin/nologin --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) > + code = subprocess.call('useradd %s' % useradd_params, shell=True) > + elif fields[0] == 'g': > + groupadd_params = "" > + if fields[1] in s_groups: > + groupadd_params += '--gid %s' % s_groups[fields[1]][2] > + bb.warn("Changing group %s's gid to (%s), verify configuration files!" % (fields[1], s_groups[fields[1]][2])) > + elif fields[2] != '-': > + groupadd_params += '--gid %s' % fields[2] > + if fields[3] != '-': > + groupadd_params += ' --comment "%s"' % (d.getVar("IMAGE_ROOTFS", d), fields[3]) > + groupadd_params += ' --root "%s" --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) > + code = subprocess.call('groupadd %s' % groupadd_params, shell=True) > +} > > # > # A hook function to support read-only-rootfs IMAGE_FEATURES > @@ -73,27 +114,6 @@ read_only_rootfs_hook () { > ${IMAGE_ROOTFS}/etc/init.d/populate-volatile.sh > fi > fi > - > - if ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "true", "false", d)}; then > - # Update user database files so that services don't fail for a read-only systemd system > - for conffile in ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd.conf ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd-remote.conf; do > - [ -e $conffile ] || continue > - grep -v "^#" $conffile | sed -e '/^$/d' | while read type name id comment; do > - if [ "$type" = "u" ]; then > - useradd_params="" > - [ "$id" != "-" ] && useradd_params="$useradd_params --uid $id" > - [ "$comment" != "-" ] && useradd_params="$useradd_params --comment $comment" > - useradd_params="$useradd_params --system $name" > - eval useradd --root ${IMAGE_ROOTFS} $useradd_params || true > - elif [ "$type" = "g" ]; then > - groupadd_params="" > - [ "$id" != "-" ] && groupadd_params="$groupadd_params --gid $id" > - groupadd_params="$groupadd_params --system $name" > - eval groupadd --root ${IMAGE_ROOTFS} $groupadd_params || true > - fi > - done > - done > - fi > } > > # > diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass > index a9b506d..e9dfe69 100644 > --- a/meta/classes/useradd-staticids.bbclass > +++ b/meta/classes/useradd-staticids.bbclass > @@ -1,3 +1,4 @@ > +inherit useradd-staticids-list > # In order to support a deterministic set of 'dynamic' users/groups, > # we need a function to reformat the params based on a static file > def update_useradd_static_config(d): > @@ -17,35 +18,6 @@ def update_useradd_static_config(d): > def error(self, message): > raise bb.build.FuncFailed(message) > > - def list_extend(iterable, length, obj = None): > - """Ensure that iterable is the specified length by extending with obj > - and return it as a list""" > - return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) > - > - def merge_files(file_list, exp_fields): > - """Read each passwd/group file in file_list, split each line and create > - a dictionary with the user/group names as keys and the split lines as > - values. If the user/group name already exists in the dictionary, then > - update any fields in the list with the values from the new list (if they > - are set).""" > - id_table = dict() > - for conf in file_list.split(): > - if os.path.exists(conf): > - f = open(conf, "r") > - for line in f: > - if line.startswith('#'): > - continue > - # Make sure there always are at least exp_fields elements in > - # the field list. This allows for leaving out trailing > - # colons in the files. > - fields = list_extend(line.rstrip().split(":"), exp_fields) > - if fields[0] not in id_table: > - id_table[fields[0]] = fields > - else: > - id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) > - > - return id_table > - > # We parse and rewrite the useradd components > def rewrite_useradd(params): > # The following comes from --help on useradd from shadow > @@ -73,19 +45,6 @@ def update_useradd_static_config(d): > parser.add_argument("-U", "--user-group", help="create a group with the same name as the user", action="store_const", const=True) > parser.add_argument("LOGIN", help="Login name of the new user") > > - # Return a list of configuration files based on either the default > - # files/passwd or the contents of USERADD_UID_TABLES > - # paths are resolved via BBPATH > - def get_passwd_list(d): > - str = "" > - bbpath = d.getVar('BBPATH', True) > - passwd_tables = d.getVar('USERADD_UID_TABLES', True) > - if not passwd_tables: > - passwd_tables = 'files/passwd' > - for conf_file in passwd_tables.split(): > - str += " %s" % bb.utils.which(bbpath, conf_file) > - return str > - > newparams = [] > users = None > for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): > @@ -206,19 +165,6 @@ def update_useradd_static_config(d): > parser.add_argument("-r", "--system", help="create a system account", action="store_true") > parser.add_argument("GROUP", help="Group name of the new group") > > - # Return a list of configuration files based on either the default > - # files/group or the contents of USERADD_GID_TABLES > - # paths are resolved via BBPATH > - def get_group_list(d): > - str = "" > - bbpath = d.getVar('BBPATH', True) > - group_tables = d.getVar('USERADD_GID_TABLES', True) > - if not group_tables: > - group_tables = 'files/group' > - for conf_file in group_tables.split(): > - str += " %s" % bb.utils.which(bbpath, conf_file) > - return str > - > newparams = [] > groups = None > for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): > diff --git a/useradd-staticids-list.bbclass b/useradd-staticids-list.bbclass > new file mode 100644 > index 0000000..377f748 > --- /dev/null > +++ b/useradd-staticids-list.bbclass > @@ -0,0 +1,56 @@ > +# Return a list of configuration files based on either the default > +# files/group or the contents of USERADD_GID_TABLES > +# paths are resolved via BBPATH > +def get_group_list(d): > + str = "" > + bbpath = d.getVar('BBPATH', True) > + group_tables = d.getVar('USERADD_GID_TABLES', True) > + if not group_tables: > + group_tables = 'files/group' > + for conf_file in group_tables.split(): > + str += " %s" % bb.utils.which(bbpath, conf_file) > + return str > + > +def list_extend(iterable, length, obj = None): > + import itertools > + """Ensure that iterable is the specified length by extending with obj > + and return it as a list""" > + return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) > + > +def merge_files(file_list, exp_fields): > + import itertools > + """Read each passwd/group file in file_list, split each line and create > + a dictionary with the user/group names as keys and the split lines as > + values. If the user/group name already exists in the dictionary, then > + update any fields in the list with the values from the new list (if they > + are set).""" > + id_table = dict() > + for conf in file_list.split(): > + if os.path.exists(conf): > + f = open(conf, "r") > + for line in f: > + if line.startswith('#'): > + continue > + # Make sure there always are at least exp_fields elements in > + # the field list. This allows for leaving out trailing > + # colons in the files. > + fields = list_extend(line.rstrip().split(":"), exp_fields) > + if fields[0] not in id_table: > + id_table[fields[0]] = fields > + else: > + id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) > + > + return id_table > + > +# Return a list of configuration files based on either the default > +# files/passwd or the contents of USERADD_UID_TABLES > +# paths are resolved via BBPATH > +def get_passwd_list(d): > + str = "" > + bbpath = d.getVar('BBPATH', True) > + passwd_tables = d.getVar('USERADD_UID_TABLES', True) > + if not passwd_tables: > + passwd_tables = 'files/passwd' > + for conf_file in passwd_tables.split(): > + str += " %s" % bb.utils.which(bbpath, conf_file) > + return str > -- > 2.8.0 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 211 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 2/2] systemd: add users/groups at build rather than runtime 2016-05-09 23:49 ` Khem Raj @ 2016-05-10 0:55 ` Stephano Cetola 2016-05-10 0:58 ` Khem Raj 0 siblings, 1 reply; 12+ messages in thread From: Stephano Cetola @ 2016-05-10 0:55 UTC (permalink / raw) To: Khem Raj; +Cc: openembedded-core On 05/09, Khem Raj wrote: > > > On May 9, 2016, at 4:42 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: > > > > This fix ensures that systemd users are created at build and that any > > staticids for the users or groups are preserved. This functionality only > > existed for readonly filesystems, however it seems wise to be sure these > > users are added rather than relying on systemd to create them. > > We need to cover the case where online package management is used. If you were to fix > it in systemd recipe itself which takes care of creating these users in post installs > that will fix it for this case too. I'm not sure I follow you. I can see how this could be done in a post install script, but I'm missing how this would help online package management. Could you please point me in the right direction? > > > > > [ YOCTO #9497 ] > > > > Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> > > --- > > meta/classes/rootfs-postcommands.bbclass | 66 +++++++++++++++++++++----------- > > meta/classes/useradd-staticids.bbclass | 56 +-------------------------- > > useradd-staticids-list.bbclass | 56 +++++++++++++++++++++++++++ > > 3 files changed, 100 insertions(+), 78 deletions(-) > > create mode 100644 useradd-staticids-list.bbclass > > > > diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass > > index 95d28af..db4cc75 100644 > > --- a/meta/classes/rootfs-postcommands.bbclass > > +++ b/meta/classes/rootfs-postcommands.bbclass > > @@ -1,4 +1,4 @@ > > - > > +inherit useradd-staticids-list > > # Zap the root password if debug-tweaks feature is not enabled > > ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'debug-tweaks', 'empty-root-password' ], "", "zap_empty_root_password ; ",d)}' > > > > @@ -23,6 +23,9 @@ POSTINST_LOGFILE ?= "${localstatedir}/log/postinstall.log" > > SYSTEMD_DEFAULT_TARGET ?= '${@bb.utils.contains("IMAGE_FEATURES", "x11-base", "graphical.target", "multi-user.target", d)}' > > ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "set_systemd_default_target; ", "", d)}' > > > > +# Create systemd users at build rather than at runtime > > +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "create_systemd_sysusers; ", "", d)}' > > + > > ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' > > > > # Disable DNS lookups, the SSH_DISABLE_DNS_LOOKUP can be overridden to allow > > @@ -30,7 +33,45 @@ ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' > > SSH_DISABLE_DNS_LOOKUP ?= " ssh_disable_dns_lookup ; " > > ROOTFS_POSTPROCESS_COMMAND_append_qemuall = "${SSH_DISABLE_DNS_LOOKUP}" > > > > - > > +# Add systemd users checking for any staticids > > +python create_systemd_sysusers () { > > + import shlex > > + import subprocess > > + s_users = merge_files(get_passwd_list(d), 7) > > + s_groups = merge_files(get_group_list(d), 4) > > + confs = ['%s/usr/lib/sysusers.d/systemd.conf' % d.getVar('IMAGE_ROOTFS', d), '%s/usr/lib/sysusers.d/systemd-remote.conf' % d.getVar('IMAGE_ROOTFS', d)] > > + for conf in confs: > > + if os.path.isfile(conf): > > + with open(conf, 'r') as f: > > + for line in f: > > + if line.startswith('#'): > > + continue > > + fields = shlex.split(line.rstrip()) > > + if not fields: > > + continue > > + if fields[0] == 'u': > > + useradd_params = '' > > + if fields[1] in s_users: > > + useradd_params += '--uid %s' % s_users[fields[1]][2] > > + bb.warn("Changing user %s's uid to (%s), verify configuration files!" % (fields[1], s_users[fields[1]][2])) > > + elif fields[2] != '-': > > + useradd_params += '--uid %s' % fields[2] > > + if fields[3] != '-': > > + useradd_params += ' --comment "%s"' % fields[3] > > + useradd_params += ' --root "%s" --shell /sbin/nologin --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) > > + code = subprocess.call('useradd %s' % useradd_params, shell=True) > > + elif fields[0] == 'g': > > + groupadd_params = "" > > + if fields[1] in s_groups: > > + groupadd_params += '--gid %s' % s_groups[fields[1]][2] > > + bb.warn("Changing group %s's gid to (%s), verify configuration files!" % (fields[1], s_groups[fields[1]][2])) > > + elif fields[2] != '-': > > + groupadd_params += '--gid %s' % fields[2] > > + if fields[3] != '-': > > + groupadd_params += ' --comment "%s"' % (d.getVar("IMAGE_ROOTFS", d), fields[3]) > > + groupadd_params += ' --root "%s" --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) > > + code = subprocess.call('groupadd %s' % groupadd_params, shell=True) > > +} > > > > # > > # A hook function to support read-only-rootfs IMAGE_FEATURES > > @@ -73,27 +114,6 @@ read_only_rootfs_hook () { > > ${IMAGE_ROOTFS}/etc/init.d/populate-volatile.sh > > fi > > fi > > - > > - if ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "true", "false", d)}; then > > - # Update user database files so that services don't fail for a read-only systemd system > > - for conffile in ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd.conf ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd-remote.conf; do > > - [ -e $conffile ] || continue > > - grep -v "^#" $conffile | sed -e '/^$/d' | while read type name id comment; do > > - if [ "$type" = "u" ]; then > > - useradd_params="" > > - [ "$id" != "-" ] && useradd_params="$useradd_params --uid $id" > > - [ "$comment" != "-" ] && useradd_params="$useradd_params --comment $comment" > > - useradd_params="$useradd_params --system $name" > > - eval useradd --root ${IMAGE_ROOTFS} $useradd_params || true > > - elif [ "$type" = "g" ]; then > > - groupadd_params="" > > - [ "$id" != "-" ] && groupadd_params="$groupadd_params --gid $id" > > - groupadd_params="$groupadd_params --system $name" > > - eval groupadd --root ${IMAGE_ROOTFS} $groupadd_params || true > > - fi > > - done > > - done > > - fi > > } > > > > # > > diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass > > index a9b506d..e9dfe69 100644 > > --- a/meta/classes/useradd-staticids.bbclass > > +++ b/meta/classes/useradd-staticids.bbclass > > @@ -1,3 +1,4 @@ > > +inherit useradd-staticids-list > > # In order to support a deterministic set of 'dynamic' users/groups, > > # we need a function to reformat the params based on a static file > > def update_useradd_static_config(d): > > @@ -17,35 +18,6 @@ def update_useradd_static_config(d): > > def error(self, message): > > raise bb.build.FuncFailed(message) > > > > - def list_extend(iterable, length, obj = None): > > - """Ensure that iterable is the specified length by extending with obj > > - and return it as a list""" > > - return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) > > - > > - def merge_files(file_list, exp_fields): > > - """Read each passwd/group file in file_list, split each line and create > > - a dictionary with the user/group names as keys and the split lines as > > - values. If the user/group name already exists in the dictionary, then > > - update any fields in the list with the values from the new list (if they > > - are set).""" > > - id_table = dict() > > - for conf in file_list.split(): > > - if os.path.exists(conf): > > - f = open(conf, "r") > > - for line in f: > > - if line.startswith('#'): > > - continue > > - # Make sure there always are at least exp_fields elements in > > - # the field list. This allows for leaving out trailing > > - # colons in the files. > > - fields = list_extend(line.rstrip().split(":"), exp_fields) > > - if fields[0] not in id_table: > > - id_table[fields[0]] = fields > > - else: > > - id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) > > - > > - return id_table > > - > > # We parse and rewrite the useradd components > > def rewrite_useradd(params): > > # The following comes from --help on useradd from shadow > > @@ -73,19 +45,6 @@ def update_useradd_static_config(d): > > parser.add_argument("-U", "--user-group", help="create a group with the same name as the user", action="store_const", const=True) > > parser.add_argument("LOGIN", help="Login name of the new user") > > > > - # Return a list of configuration files based on either the default > > - # files/passwd or the contents of USERADD_UID_TABLES > > - # paths are resolved via BBPATH > > - def get_passwd_list(d): > > - str = "" > > - bbpath = d.getVar('BBPATH', True) > > - passwd_tables = d.getVar('USERADD_UID_TABLES', True) > > - if not passwd_tables: > > - passwd_tables = 'files/passwd' > > - for conf_file in passwd_tables.split(): > > - str += " %s" % bb.utils.which(bbpath, conf_file) > > - return str > > - > > newparams = [] > > users = None > > for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): > > @@ -206,19 +165,6 @@ def update_useradd_static_config(d): > > parser.add_argument("-r", "--system", help="create a system account", action="store_true") > > parser.add_argument("GROUP", help="Group name of the new group") > > > > - # Return a list of configuration files based on either the default > > - # files/group or the contents of USERADD_GID_TABLES > > - # paths are resolved via BBPATH > > - def get_group_list(d): > > - str = "" > > - bbpath = d.getVar('BBPATH', True) > > - group_tables = d.getVar('USERADD_GID_TABLES', True) > > - if not group_tables: > > - group_tables = 'files/group' > > - for conf_file in group_tables.split(): > > - str += " %s" % bb.utils.which(bbpath, conf_file) > > - return str > > - > > newparams = [] > > groups = None > > for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): > > diff --git a/useradd-staticids-list.bbclass b/useradd-staticids-list.bbclass > > new file mode 100644 > > index 0000000..377f748 > > --- /dev/null > > +++ b/useradd-staticids-list.bbclass > > @@ -0,0 +1,56 @@ > > +# Return a list of configuration files based on either the default > > +# files/group or the contents of USERADD_GID_TABLES > > +# paths are resolved via BBPATH > > +def get_group_list(d): > > + str = "" > > + bbpath = d.getVar('BBPATH', True) > > + group_tables = d.getVar('USERADD_GID_TABLES', True) > > + if not group_tables: > > + group_tables = 'files/group' > > + for conf_file in group_tables.split(): > > + str += " %s" % bb.utils.which(bbpath, conf_file) > > + return str > > + > > +def list_extend(iterable, length, obj = None): > > + import itertools > > + """Ensure that iterable is the specified length by extending with obj > > + and return it as a list""" > > + return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) > > + > > +def merge_files(file_list, exp_fields): > > + import itertools > > + """Read each passwd/group file in file_list, split each line and create > > + a dictionary with the user/group names as keys and the split lines as > > + values. If the user/group name already exists in the dictionary, then > > + update any fields in the list with the values from the new list (if they > > + are set).""" > > + id_table = dict() > > + for conf in file_list.split(): > > + if os.path.exists(conf): > > + f = open(conf, "r") > > + for line in f: > > + if line.startswith('#'): > > + continue > > + # Make sure there always are at least exp_fields elements in > > + # the field list. This allows for leaving out trailing > > + # colons in the files. > > + fields = list_extend(line.rstrip().split(":"), exp_fields) > > + if fields[0] not in id_table: > > + id_table[fields[0]] = fields > > + else: > > + id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) > > + > > + return id_table > > + > > +# Return a list of configuration files based on either the default > > +# files/passwd or the contents of USERADD_UID_TABLES > > +# paths are resolved via BBPATH > > +def get_passwd_list(d): > > + str = "" > > + bbpath = d.getVar('BBPATH', True) > > + passwd_tables = d.getVar('USERADD_UID_TABLES', True) > > + if not passwd_tables: > > + passwd_tables = 'files/passwd' > > + for conf_file in passwd_tables.split(): > > + str += " %s" % bb.utils.which(bbpath, conf_file) > > + return str > > -- > > 2.8.0 > > > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 2/2] systemd: add users/groups at build rather than runtime 2016-05-10 0:55 ` Stephano Cetola @ 2016-05-10 0:58 ` Khem Raj 0 siblings, 0 replies; 12+ messages in thread From: Khem Raj @ 2016-05-10 0:58 UTC (permalink / raw) To: Stephano Cetola; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 14594 bytes --] > On May 9, 2016, at 5:55 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: > > On 05/09, Khem Raj wrote: >> >>> On May 9, 2016, at 4:42 PM, Stephano Cetola <stephano.cetola@linux.intel.com> wrote: >>> >>> This fix ensures that systemd users are created at build and that any >>> staticids for the users or groups are preserved. This functionality only >>> existed for readonly filesystems, however it seems wise to be sure these >>> users are added rather than relying on systemd to create them. >> >> We need to cover the case where online package management is used. If you were to fix >> it in systemd recipe itself which takes care of creating these users in post installs >> that will fix it for this case too. > I'm not sure I follow you. I can see how this could be done in a post > install script, but I'm missing how this would help online package > management. Could you please point me in the right direction? Lets say, I did not install microhttpserver in default image. But later on I decided to install it via O_P_M feeds. or remove for that matter. >> >>> >>> [ YOCTO #9497 ] >>> >>> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> >>> --- >>> meta/classes/rootfs-postcommands.bbclass | 66 +++++++++++++++++++++----------- >>> meta/classes/useradd-staticids.bbclass | 56 +-------------------------- >>> useradd-staticids-list.bbclass | 56 +++++++++++++++++++++++++++ >>> 3 files changed, 100 insertions(+), 78 deletions(-) >>> create mode 100644 useradd-staticids-list.bbclass >>> >>> diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass >>> index 95d28af..db4cc75 100644 >>> --- a/meta/classes/rootfs-postcommands.bbclass >>> +++ b/meta/classes/rootfs-postcommands.bbclass >>> @@ -1,4 +1,4 @@ >>> - >>> +inherit useradd-staticids-list >>> # Zap the root password if debug-tweaks feature is not enabled >>> ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'debug-tweaks', 'empty-root-password' ], "", "zap_empty_root_password ; ",d)}' >>> >>> @@ -23,6 +23,9 @@ POSTINST_LOGFILE ?= "${localstatedir}/log/postinstall.log" >>> SYSTEMD_DEFAULT_TARGET ?= '${@bb.utils.contains("IMAGE_FEATURES", "x11-base", "graphical.target", "multi-user.target", d)}' >>> ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "set_systemd_default_target; ", "", d)}' >>> >>> +# Create systemd users at build rather than at runtime >>> +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "create_systemd_sysusers; ", "", d)}' >>> + >>> ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' >>> >>> # Disable DNS lookups, the SSH_DISABLE_DNS_LOOKUP can be overridden to allow >>> @@ -30,7 +33,45 @@ ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;' >>> SSH_DISABLE_DNS_LOOKUP ?= " ssh_disable_dns_lookup ; " >>> ROOTFS_POSTPROCESS_COMMAND_append_qemuall = "${SSH_DISABLE_DNS_LOOKUP}" >>> >>> - >>> +# Add systemd users checking for any staticids >>> +python create_systemd_sysusers () { >>> + import shlex >>> + import subprocess >>> + s_users = merge_files(get_passwd_list(d), 7) >>> + s_groups = merge_files(get_group_list(d), 4) >>> + confs = ['%s/usr/lib/sysusers.d/systemd.conf' % d.getVar('IMAGE_ROOTFS', d), '%s/usr/lib/sysusers.d/systemd-remote.conf' % d.getVar('IMAGE_ROOTFS', d)] >>> + for conf in confs: >>> + if os.path.isfile(conf): >>> + with open(conf, 'r') as f: >>> + for line in f: >>> + if line.startswith('#'): >>> + continue >>> + fields = shlex.split(line.rstrip()) >>> + if not fields: >>> + continue >>> + if fields[0] == 'u': >>> + useradd_params = '' >>> + if fields[1] in s_users: >>> + useradd_params += '--uid %s' % s_users[fields[1]][2] >>> + bb.warn("Changing user %s's uid to (%s), verify configuration files!" % (fields[1], s_users[fields[1]][2])) >>> + elif fields[2] != '-': >>> + useradd_params += '--uid %s' % fields[2] >>> + if fields[3] != '-': >>> + useradd_params += ' --comment "%s"' % fields[3] >>> + useradd_params += ' --root "%s" --shell /sbin/nologin --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) >>> + code = subprocess.call('useradd %s' % useradd_params, shell=True) >>> + elif fields[0] == 'g': >>> + groupadd_params = "" >>> + if fields[1] in s_groups: >>> + groupadd_params += '--gid %s' % s_groups[fields[1]][2] >>> + bb.warn("Changing group %s's gid to (%s), verify configuration files!" % (fields[1], s_groups[fields[1]][2])) >>> + elif fields[2] != '-': >>> + groupadd_params += '--gid %s' % fields[2] >>> + if fields[3] != '-': >>> + groupadd_params += ' --comment "%s"' % (d.getVar("IMAGE_ROOTFS", d), fields[3]) >>> + groupadd_params += ' --root "%s" --system %s' % (d.getVar("IMAGE_ROOTFS", d), fields[1]) >>> + code = subprocess.call('groupadd %s' % groupadd_params, shell=True) >>> +} >>> >>> # >>> # A hook function to support read-only-rootfs IMAGE_FEATURES >>> @@ -73,27 +114,6 @@ read_only_rootfs_hook () { >>> ${IMAGE_ROOTFS}/etc/init.d/populate-volatile.sh >>> fi >>> fi >>> - >>> - if ${@bb.utils.contains("DISTRO_FEATURES", "systemd", "true", "false", d)}; then >>> - # Update user database files so that services don't fail for a read-only systemd system >>> - for conffile in ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd.conf ${IMAGE_ROOTFS}/usr/lib/sysusers.d/systemd-remote.conf; do >>> - [ -e $conffile ] || continue >>> - grep -v "^#" $conffile | sed -e '/^$/d' | while read type name id comment; do >>> - if [ "$type" = "u" ]; then >>> - useradd_params="" >>> - [ "$id" != "-" ] && useradd_params="$useradd_params --uid $id" >>> - [ "$comment" != "-" ] && useradd_params="$useradd_params --comment $comment" >>> - useradd_params="$useradd_params --system $name" >>> - eval useradd --root ${IMAGE_ROOTFS} $useradd_params || true >>> - elif [ "$type" = "g" ]; then >>> - groupadd_params="" >>> - [ "$id" != "-" ] && groupadd_params="$groupadd_params --gid $id" >>> - groupadd_params="$groupadd_params --system $name" >>> - eval groupadd --root ${IMAGE_ROOTFS} $groupadd_params || true >>> - fi >>> - done >>> - done >>> - fi >>> } >>> >>> # >>> diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass >>> index a9b506d..e9dfe69 100644 >>> --- a/meta/classes/useradd-staticids.bbclass >>> +++ b/meta/classes/useradd-staticids.bbclass >>> @@ -1,3 +1,4 @@ >>> +inherit useradd-staticids-list >>> # In order to support a deterministic set of 'dynamic' users/groups, >>> # we need a function to reformat the params based on a static file >>> def update_useradd_static_config(d): >>> @@ -17,35 +18,6 @@ def update_useradd_static_config(d): >>> def error(self, message): >>> raise bb.build.FuncFailed(message) >>> >>> - def list_extend(iterable, length, obj = None): >>> - """Ensure that iterable is the specified length by extending with obj >>> - and return it as a list""" >>> - return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) >>> - >>> - def merge_files(file_list, exp_fields): >>> - """Read each passwd/group file in file_list, split each line and create >>> - a dictionary with the user/group names as keys and the split lines as >>> - values. If the user/group name already exists in the dictionary, then >>> - update any fields in the list with the values from the new list (if they >>> - are set).""" >>> - id_table = dict() >>> - for conf in file_list.split(): >>> - if os.path.exists(conf): >>> - f = open(conf, "r") >>> - for line in f: >>> - if line.startswith('#'): >>> - continue >>> - # Make sure there always are at least exp_fields elements in >>> - # the field list. This allows for leaving out trailing >>> - # colons in the files. >>> - fields = list_extend(line.rstrip().split(":"), exp_fields) >>> - if fields[0] not in id_table: >>> - id_table[fields[0]] = fields >>> - else: >>> - id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) >>> - >>> - return id_table >>> - >>> # We parse and rewrite the useradd components >>> def rewrite_useradd(params): >>> # The following comes from --help on useradd from shadow >>> @@ -73,19 +45,6 @@ def update_useradd_static_config(d): >>> parser.add_argument("-U", "--user-group", help="create a group with the same name as the user", action="store_const", const=True) >>> parser.add_argument("LOGIN", help="Login name of the new user") >>> >>> - # Return a list of configuration files based on either the default >>> - # files/passwd or the contents of USERADD_UID_TABLES >>> - # paths are resolved via BBPATH >>> - def get_passwd_list(d): >>> - str = "" >>> - bbpath = d.getVar('BBPATH', True) >>> - passwd_tables = d.getVar('USERADD_UID_TABLES', True) >>> - if not passwd_tables: >>> - passwd_tables = 'files/passwd' >>> - for conf_file in passwd_tables.split(): >>> - str += " %s" % bb.utils.which(bbpath, conf_file) >>> - return str >>> - >>> newparams = [] >>> users = None >>> for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): >>> @@ -206,19 +165,6 @@ def update_useradd_static_config(d): >>> parser.add_argument("-r", "--system", help="create a system account", action="store_true") >>> parser.add_argument("GROUP", help="Group name of the new group") >>> >>> - # Return a list of configuration files based on either the default >>> - # files/group or the contents of USERADD_GID_TABLES >>> - # paths are resolved via BBPATH >>> - def get_group_list(d): >>> - str = "" >>> - bbpath = d.getVar('BBPATH', True) >>> - group_tables = d.getVar('USERADD_GID_TABLES', True) >>> - if not group_tables: >>> - group_tables = 'files/group' >>> - for conf_file in group_tables.split(): >>> - str += " %s" % bb.utils.which(bbpath, conf_file) >>> - return str >>> - >>> newparams = [] >>> groups = None >>> for param in re.split('''[ \t]*;[ \t]*(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''', params): >>> diff --git a/useradd-staticids-list.bbclass b/useradd-staticids-list.bbclass >>> new file mode 100644 >>> index 0000000..377f748 >>> --- /dev/null >>> +++ b/useradd-staticids-list.bbclass >>> @@ -0,0 +1,56 @@ >>> +# Return a list of configuration files based on either the default >>> +# files/group or the contents of USERADD_GID_TABLES >>> +# paths are resolved via BBPATH >>> +def get_group_list(d): >>> + str = "" >>> + bbpath = d.getVar('BBPATH', True) >>> + group_tables = d.getVar('USERADD_GID_TABLES', True) >>> + if not group_tables: >>> + group_tables = 'files/group' >>> + for conf_file in group_tables.split(): >>> + str += " %s" % bb.utils.which(bbpath, conf_file) >>> + return str >>> + >>> +def list_extend(iterable, length, obj = None): >>> + import itertools >>> + """Ensure that iterable is the specified length by extending with obj >>> + and return it as a list""" >>> + return list(itertools.islice(itertools.chain(iterable, itertools.repeat(obj)), length)) >>> + >>> +def merge_files(file_list, exp_fields): >>> + import itertools >>> + """Read each passwd/group file in file_list, split each line and create >>> + a dictionary with the user/group names as keys and the split lines as >>> + values. If the user/group name already exists in the dictionary, then >>> + update any fields in the list with the values from the new list (if they >>> + are set).""" >>> + id_table = dict() >>> + for conf in file_list.split(): >>> + if os.path.exists(conf): >>> + f = open(conf, "r") >>> + for line in f: >>> + if line.startswith('#'): >>> + continue >>> + # Make sure there always are at least exp_fields elements in >>> + # the field list. This allows for leaving out trailing >>> + # colons in the files. >>> + fields = list_extend(line.rstrip().split(":"), exp_fields) >>> + if fields[0] not in id_table: >>> + id_table[fields[0]] = fields >>> + else: >>> + id_table[fields[0]] = list(itertools.imap(lambda x, y: x or y, fields, id_table[fields[0]])) >>> + >>> + return id_table >>> + >>> +# Return a list of configuration files based on either the default >>> +# files/passwd or the contents of USERADD_UID_TABLES >>> +# paths are resolved via BBPATH >>> +def get_passwd_list(d): >>> + str = "" >>> + bbpath = d.getVar('BBPATH', True) >>> + passwd_tables = d.getVar('USERADD_UID_TABLES', True) >>> + if not passwd_tables: >>> + passwd_tables = 'files/passwd' >>> + for conf_file in passwd_tables.split(): >>> + str += " %s" % bb.utils.which(bbpath, conf_file) >>> + return str >>> -- >>> 2.8.0 >>> >>> -- >>> _______________________________________________ >>> Openembedded-core mailing list >>> Openembedded-core@lists.openembedded.org >>> http://lists.openembedded.org/mailman/listinfo/openembedded-core >> > > [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 211 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/2] Create systemd users at build time 2016-05-09 23:42 [PATCH v2 0/2] Create systemd users at build time Stephano Cetola 2016-05-09 23:42 ` [PATCH v2 1/2] systemd: add sysusers config option to defaults Stephano Cetola 2016-05-09 23:42 ` [PATCH v2 2/2] systemd: add users/groups at build rather than runtime Stephano Cetola @ 2016-05-11 10:46 ` Puustinen, Ismo 2016-05-11 19:04 ` Stephano Cetola 2 siblings, 1 reply; 12+ messages in thread From: Puustinen, Ismo @ 2016-05-11 10:46 UTC (permalink / raw) To: openembedded-core@lists.openembedded.org, stephano.cetola@linux.intel.com On Mon, 2016-05-09 at 16:42 -0700, Stephano Cetola wrote: > Systemd has a config option sysusers which creates any users or > groups that > systemd needs on first boot. There is a rootfs-postcommand that > creates these > users in case the filesystem is readonly, however it does not take > into > account any staicids that the user may have set. > > By adding these users at build time, we allow a check for staticids, > which would > otherwise have been skipped. I think this is a valuable feature. However, have you considered adding the "m" and "r" options too? I think that at least "m" would be worthwhile to implement, since it's important to be able to add users to different groups for privilege management. Ismo ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/2] Create systemd users at build time 2016-05-11 10:46 ` [PATCH v2 0/2] Create systemd users at build time Puustinen, Ismo @ 2016-05-11 19:04 ` Stephano Cetola 2016-05-12 11:17 ` Puustinen, Ismo 0 siblings, 1 reply; 12+ messages in thread From: Stephano Cetola @ 2016-05-11 19:04 UTC (permalink / raw) To: Puustinen, Ismo; +Cc: openembedded-core@lists.openembedded.org On 05/11, Puustinen, Ismo wrote: > On Mon, 2016-05-09 at 16:42 -0700, Stephano Cetola wrote: > > Systemd has a config option sysusers which creates any users or > > groups that > > systemd needs on first boot. There is a rootfs-postcommand that > > creates these > > users in case the filesystem is readonly, however it does not take > > into > > account any staicids that the user may have set. > > > > By adding these users at build time, we allow a check for staticids, > > which would > > otherwise have been skipped. > > I think this is a valuable feature. However, have you considered adding > the "m" and "r" options too? I think that at least "m" would be > worthwhile to implement, since it's important to be able to add users > to different groups for privilege management. > > Ismo I added "--system" to both useradd and groupadd, and I believe that is the same as "-r". Please correct me if I am wrong there. I do not see how the "-m" option would help with groups. I thought that simply added a skeleton home directory. Could you give me an example? Stephano ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/2] Create systemd users at build time 2016-05-11 19:04 ` Stephano Cetola @ 2016-05-12 11:17 ` Puustinen, Ismo 2016-05-12 15:30 ` Stephano Cetola 0 siblings, 1 reply; 12+ messages in thread From: Puustinen, Ismo @ 2016-05-12 11:17 UTC (permalink / raw) To: stephano.cetola@linux.intel.com; +Cc: openembedded-core@lists.openembedded.org On Wed, 2016-05-11 at 12:04 -0700, Stephano Cetola wrote: > > I think this is a valuable feature. However, have you considered > > adding > > the "m" and "r" options too? I think that at least "m" would be > > worthwhile to implement, since it's important to be able to add > > users > > to different groups for privilege management. > > I added "--system" to both useradd and groupadd, and I believe that > is > the same as "-r". Please correct me if I am wrong there. > > I do not see how the "-m" option would help with groups. I thought > that > simply added a skeleton home directory. Could you give me an example? I mean that when you parse the sysusers.d configuration file in create_systemd_sysusers() function, you only parse the lines starting with 'u' and 'g'. You are skipping 'm' and 'r'. See: https://www.freedesktop.org/software/systemd/man/sysusers.d.html What I'm after is a generic mechanism for reading all sysusers.d/ configuration files during image creation and creating the necessary users and groups during that time. You could also separate mechanism and policy, meaning that the list of files to parse could come from outside of the create_systemd_sysusers() function. Ismo ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/2] Create systemd users at build time 2016-05-12 11:17 ` Puustinen, Ismo @ 2016-05-12 15:30 ` Stephano Cetola 0 siblings, 0 replies; 12+ messages in thread From: Stephano Cetola @ 2016-05-12 15:30 UTC (permalink / raw) To: Puustinen, Ismo; +Cc: openembedded-core@lists.openembedded.org On 05/12, Puustinen, Ismo wrote: > On Wed, 2016-05-11 at 12:04 -0700, Stephano Cetola wrote: > > > I think this is a valuable feature. However, have you considered > > > adding > > > the "m" and "r" options too? I think that at least "m" would be > > > worthwhile to implement, since it's important to be able to add > > > users > > > to different groups for privilege management. > > > > I added "--system" to both useradd and groupadd, and I believe that > > is > > the same as "-r". Please correct me if I am wrong there. > > > > I do not see how the "-m" option would help with groups. I thought > > that > > simply added a skeleton home directory. Could you give me an example? > > I mean that when you parse the sysusers.d configuration file > in create_systemd_sysusers() function, you only parse the lines > starting with 'u' and 'g'. You are skipping 'm' and 'r'. See: > > https://www.freedesktop.org/software/systemd/man/sysusers.d.html > > What I'm after is a generic mechanism for reading all sysusers.d/ > configuration files during image creation and creating the necessary > users and groups during that time. You could also separate mechanism > and policy, meaning that the list of files to parse could come from > outside of the create_systemd_sysusers() function. > > Ismo Ah, I see. Yes, I agree this would be of value. I think that "m" should be straight forward to implement, though "r" might get a little complicated with the staticids class. I'll look into both of these. Cheers, Stephano ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2016-05-12 15:30 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-05-09 23:42 [PATCH v2 0/2] Create systemd users at build time Stephano Cetola 2016-05-09 23:42 ` [PATCH v2 1/2] systemd: add sysusers config option to defaults Stephano Cetola 2016-05-09 23:50 ` Khem Raj 2016-05-10 0:24 ` Stephano Cetola 2016-05-09 23:42 ` [PATCH v2 2/2] systemd: add users/groups at build rather than runtime Stephano Cetola 2016-05-09 23:49 ` Khem Raj 2016-05-10 0:55 ` Stephano Cetola 2016-05-10 0:58 ` Khem Raj 2016-05-11 10:46 ` [PATCH v2 0/2] Create systemd users at build time Puustinen, Ismo 2016-05-11 19:04 ` Stephano Cetola 2016-05-12 11:17 ` Puustinen, Ismo 2016-05-12 15:30 ` Stephano Cetola
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox