* [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-15 1:59 [PATCH 0/1] Replace retry logic in useradd " kai.kang
@ 2016-02-15 1:59 ` kai.kang
2016-02-15 12:31 ` Burton, Ross
0 siblings, 1 reply; 18+ messages in thread
From: kai.kang @ 2016-02-15 1:59 UTC (permalink / raw)
To: ross.burton; +Cc: openembedded-core
From: Kai Kang <kai.kang@windriver.com>
When perform useradd during populate sysroot, it locks files passwd.lock
and group.lock at same time. And then it meets a dead lock issue.
Use flock to reslove it by using an universal lock file for all the
user and group related operations. The lock file is put in image dir
/tmp. /tmp will be mounted as a tmpfs, so the lock file will not be
visible when boot the target.
[YOCTO #9022]
Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
meta/classes/useradd.bbclass | 8 +-
meta/classes/useradd_base.bbclass | 165 ++++++++++----------------------------
2 files changed, 48 insertions(+), 125 deletions(-)
diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index c960656..db74123 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -48,6 +48,8 @@ if test "x$UA_SYSROOT" = "x"; then
GROUPMEMS_PARAM="${GROUPMEMS_PARAM}"
fi
+mkdir -p $SYSROOT/tmp
+
# Perform group additions first, since user additions may depend
# on these groups existing
if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
@@ -57,7 +59,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupadd "$SYSROOT" "$OPT $opts" 10
+ perform_groupadd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -73,7 +75,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_useradd "$SYSROOT" "$OPT $opts" 10
+ perform_useradd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -89,7 +91,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupmems "$SYSROOT" "$OPT $opts" 10
+ perform_groupmems "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index ab3cd35..a5baceb 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -4,7 +4,7 @@
# The following functions basically have similar logic.
# *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
# *) Error out if an error occurs.
# Note that before invoking these functions, make sure the global variable
@@ -13,26 +13,16 @@
perform_groupadd () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupadd with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupadd $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" = "x"; then
- bbwarn "${PN}: groupadd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
- fi
- sleep $count
- done
+ opts=`echo $opts | sed s/\'/\"/g`
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO groupadd $opts\' || true
+ group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ if test "x$group_exists" = "x"; then
+ bbfatal "${PN}: groupadd command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname already exists, not re-creating it"
fi
@@ -41,26 +31,16 @@ perform_groupadd () {
perform_useradd () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing useradd with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO useradd $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" = "x"; then
- bbwarn "${PN}: useradd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
- fi
- sleep $count
- done
+ opts=`echo $opts | sed s/\'/\"/g`
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO useradd $opts\' || true
+ user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ if test "x$user_exists" = "x"; then
+ bbfatal "${PN}: useradd command did not succeed."
+ fi
else
bbnote "${PN}: user $username already exists, not re-creating it"
fi
@@ -69,8 +49,7 @@ perform_useradd () {
perform_groupmems () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupmems with [$opts]"
local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
bbnote "${PN}: Running groupmems command with group $groupname and user $username"
@@ -84,25 +63,11 @@ perform_groupmems () {
fi
local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
if test "x$mem_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmems $opts || true
- mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
- if test "x$mem_exists" = "x"; then
- bbwarn "${PN}: groupmems command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- if test "x$gshadow" = "xno"; then
- rm -f $rootdir${sysconfdir}/gshadow
- rm -f $rootdir${sysconfdir}/gshadow-
- fi
- bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO groupmems $opts\' || true
+ mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+ if test "x$mem_exists" = "x"; then
+ bbfatal "${PN}: groupmems command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname already contains $username, not re-adding it"
fi
@@ -115,26 +80,15 @@ perform_groupmems () {
perform_groupdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupdel with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupdel $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" != "x"; then
- bbwarn "${PN}: groupdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO groupdel $opts\' || true
+ group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ if test "x$group_exists" != "x"; then
+ bbfatal "${PN}: groupdel command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname doesn't exist, not removing it"
fi
@@ -143,26 +97,15 @@ perform_groupdel () {
perform_userdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing userdel with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO userdel $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" != "x"; then
- bbwarn "${PN}: userdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO userdel $opts\' || true
+ user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ if test "x$user_exists" != "x"; then
+ bbfatal "${PN}: userdel command did not succeed."
+ fi
else
bbnote "${PN}: user $username doesn't exist, not removing it"
fi
@@ -174,25 +117,14 @@ perform_groupmod () {
set +e
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupmod with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmod $opts
- if test $? != 0; then
- bbwarn "${PN}: groupmod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO groupmod $opts\'
+ if test $? != 0; then
+ bbwarn "${PN}: groupmod command did not succeed."
+ fi
else
bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
fi
@@ -204,25 +136,14 @@ perform_usermod () {
set +e
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing usermod with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO usermod $opts
- if test $? != 0; then
- bbwarn "${PN}: usermod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO usermod $opts\'
+ if test $? != 0; then
+ bbfatal "${PN}: usermod command did not succeed."
+ fi
else
bbwarn "${PN}: user $username doesn't exist, unable to modify it"
fi
--
2.6.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-15 1:59 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
@ 2016-02-15 12:31 ` Burton, Ross
2016-02-16 2:14 ` Kang Kai
0 siblings, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-15 12:31 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 369 bytes --]
On 15 February 2016 at 01:59, <kai.kang@windriver.com> wrote:
> + eval flock -x -w 100 $rootdir/tmp/user-and-group-ops.flock
> -c \'$PSEUDO useradd $opts\' || true
>
This is a great patch, but why not just flock $rootdir/etc instead of new
file in tmp? Surely images end up being created with
/tmp/user-and-group-ops.flock in them?
Ross
[-- Attachment #2: Type: text/html, Size: 793 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-15 12:31 ` Burton, Ross
@ 2016-02-16 2:14 ` Kang Kai
0 siblings, 0 replies; 18+ messages in thread
From: Kang Kai @ 2016-02-16 2:14 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
On 2016年02月15日 20:31, Burton, Ross wrote:
>
> On 15 February 2016 at 01:59, <kai.kang@windriver.com
> <mailto:kai.kang@windriver.com>> wrote:
>
> + eval flock -x -w 100
> $rootdir/tmp/user-and-group-ops.flock -c \'$PSEUDO useradd
> $opts\' || true
>
>
> This is a great patch, but why not just flock $rootdir/etc instead of
> new file in tmp? Surely images end up being created with
> /tmp/user-and-group-ops.flock in them?
I didn't try to think take a directory as a lock and I'll try it. For
the lock file in $rootdir/tmp/, it is ok that /tmp will be remounted and
user won't see the lock file.
Thanks,
Kai
>
> Ross
--
Regards,
Neil | Kai Kang
[-- Attachment #2: Type: text/html, Size: 2127 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-16 7:14 [PATCH 0/1] V2: Replace retry logic in useradd " kai.kang
@ 2016-02-16 7:14 ` kai.kang
2016-02-16 12:09 ` Burton, Ross
0 siblings, 1 reply; 18+ messages in thread
From: kai.kang @ 2016-02-16 7:14 UTC (permalink / raw)
To: ross.burton; +Cc: openembedded-core
From: Kai Kang <kai.kang@windriver.com>
When perform useradd during populate sysroot, it locks files passwd.lock
and group.lock at same time. And then it meets a dead lock issue
randomly.
Use flock to reslove it by using an universal lock file for all the
user and group related operations.
[YOCTO #9022]
Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
meta/classes/useradd.bbclass | 8 +-
meta/classes/useradd_base.bbclass | 165 ++++++++++----------------------------
2 files changed, 48 insertions(+), 125 deletions(-)
diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index c960656..4503020 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -38,6 +38,8 @@ if test "x$D" != "x"; then
export PSEUDO_PASSWD="$SYSROOT:${STAGING_DIR_NATIVE}"
fi
+FLOCK_FILE=$SYSROOT/etc
+
# If we're not doing a special SSTATE/SYSROOT install
# then set the values, otherwise use the environment
if test "x$UA_SYSROOT" = "x"; then
@@ -57,7 +59,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupadd "$SYSROOT" "$OPT $opts" 10
+ perform_groupadd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -73,7 +75,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_useradd "$SYSROOT" "$OPT $opts" 10
+ perform_useradd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -89,7 +91,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupmems "$SYSROOT" "$OPT $opts" 10
+ perform_groupmems "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index ab3cd35..66a9848 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -4,7 +4,7 @@
# The following functions basically have similar logic.
# *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
# *) Error out if an error occurs.
# Note that before invoking these functions, make sure the global variable
@@ -13,26 +13,16 @@
perform_groupadd () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupadd with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupadd $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" = "x"; then
- bbwarn "${PN}: groupadd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
- fi
- sleep $count
- done
+ opts=`echo $opts | sed s/\'/\"/g`
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO groupadd $opts\' || true
+ group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ if test "x$group_exists" = "x"; then
+ bbfatal "${PN}: groupadd command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname already exists, not re-creating it"
fi
@@ -41,26 +31,16 @@ perform_groupadd () {
perform_useradd () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing useradd with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO useradd $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" = "x"; then
- bbwarn "${PN}: useradd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
- fi
- sleep $count
- done
+ opts=`echo $opts | sed s/\'/\"/g`
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO useradd $opts\' || true
+ user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ if test "x$user_exists" = "x"; then
+ bbfatal "${PN}: useradd command did not succeed."
+ fi
else
bbnote "${PN}: user $username already exists, not re-creating it"
fi
@@ -69,8 +49,7 @@ perform_useradd () {
perform_groupmems () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupmems with [$opts]"
local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
bbnote "${PN}: Running groupmems command with group $groupname and user $username"
@@ -84,25 +63,11 @@ perform_groupmems () {
fi
local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
if test "x$mem_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmems $opts || true
- mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
- if test "x$mem_exists" = "x"; then
- bbwarn "${PN}: groupmems command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- if test "x$gshadow" = "xno"; then
- rm -f $rootdir${sysconfdir}/gshadow
- rm -f $rootdir${sysconfdir}/gshadow-
- fi
- bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO groupmems $opts\' || true
+ mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+ if test "x$mem_exists" = "x"; then
+ bbfatal "${PN}: groupmems command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname already contains $username, not re-adding it"
fi
@@ -115,26 +80,15 @@ perform_groupmems () {
perform_groupdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupdel with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupdel $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" != "x"; then
- bbwarn "${PN}: groupdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO groupdel $opts\' || true
+ group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ if test "x$group_exists" != "x"; then
+ bbfatal "${PN}: groupdel command did not succeed."
+ fi
else
bbnote "${PN}: group $groupname doesn't exist, not removing it"
fi
@@ -143,26 +97,15 @@ perform_groupdel () {
perform_userdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing userdel with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO userdel $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" != "x"; then
- bbwarn "${PN}: userdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO userdel $opts\' || true
+ user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ if test "x$user_exists" != "x"; then
+ bbfatal "${PN}: userdel command did not succeed."
+ fi
else
bbnote "${PN}: user $username doesn't exist, not removing it"
fi
@@ -174,25 +117,14 @@ perform_groupmod () {
set +e
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupmod with [$opts]"
local groupname=`echo "$opts" | awk '{ print $NF }'`
local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmod $opts
- if test $? != 0; then
- bbwarn "${PN}: groupmod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO groupmod $opts\'
+ if test $? != 0; then
+ bbwarn "${PN}: groupmod command did not succeed."
+ fi
else
bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
fi
@@ -204,25 +136,14 @@ perform_usermod () {
set +e
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing usermod with [$opts]"
local username=`echo "$opts" | awk '{ print $NF }'`
local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO usermod $opts
- if test $? != 0; then
- bbwarn "${PN}: usermod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
- fi
- sleep $count
- done
+ eval flock -x -w 100 $FLOCK_FILE -c \'$PSEUDO usermod $opts\'
+ if test $? != 0; then
+ bbfatal "${PN}: usermod command did not succeed."
+ fi
else
bbwarn "${PN}: user $username doesn't exist, unable to modify it"
fi
--
2.7.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-16 7:14 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
@ 2016-02-16 12:09 ` Burton, Ross
2016-02-17 5:40 ` Kang Kai
0 siblings, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-16 12:09 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 621 bytes --]
On 16 February 2016 at 07:14, <kai.kang@windriver.com> wrote:
> +FLOCK_FILE=$SYSROOT/etc
>
useradd_base is also used by extrausers.bbclass, so it would be best if
useradd_base decided where to use as a lock ($rootdir/${sysconfdir} sounds
sensible) instead of expecting that the caller sets FLOCK_FILE and
mysteriously failing if it wasn't set.
Also if we're locking and waiting properly instead of using timeouts and
loops, can we remove the "does it exist?" check before the operation (eg
groupadd -f will exit 0 if the group exists) and after (with proper locking
that shouldn't be happening).
Ross
[-- Attachment #2: Type: text/html, Size: 1109 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-16 12:09 ` Burton, Ross
@ 2016-02-17 5:40 ` Kang Kai
2016-02-17 23:49 ` Burton, Ross
0 siblings, 1 reply; 18+ messages in thread
From: Kang Kai @ 2016-02-17 5:40 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 985 bytes --]
On 2016年02月16日 20:09, Burton, Ross wrote:
>
> On 16 February 2016 at 07:14, <kai.kang@windriver.com
> <mailto:kai.kang@windriver.com>> wrote:
>
> +FLOCK_FILE=$SYSROOT/etc
>
>
> useradd_base is also used by extrausers.bbclass, so it would be best
> if useradd_base decided where to use as a lock ($rootdir/${sysconfdir}
> sounds sensible) instead of expecting that the caller sets FLOCK_FILE
> and mysteriously failing if it wasn't set.
OK. Sounds good.
>
> Also if we're locking and waiting properly instead of using timeouts
> and loops, can we remove the "does it exist?" check before the
> operation (eg groupadd -f will exit 0 if the group exists) and after
> (with proper locking that shouldn't be happening).
I prefer to keep such check. There is more than one case that causes
operation error such as pass wrong parameters. The messages are helpful
for developers.
Thanks,
Kai
>
> Ross
--
Regards,
Neil | Kai Kang
[-- Attachment #2: Type: text/html, Size: 2656 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-17 5:40 ` Kang Kai
@ 2016-02-17 23:49 ` Burton, Ross
2016-02-22 8:41 ` Kang Kai
0 siblings, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-17 23:49 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 425 bytes --]
On 17 February 2016 at 05:40, Kang Kai <Kai.Kang@windriver.com> wrote:
> I prefer to keep such check. There is more than one case that causes
> operation error such as pass wrong parameters. The messages are helpful
> for developers.
>
In that case the groupadd will return an error code. What situations with
proper locking will result in groupadd running successfully but the group
not being added?
Ross
[-- Attachment #2: Type: text/html, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-17 23:49 ` Burton, Ross
@ 2016-02-22 8:41 ` Kang Kai
0 siblings, 0 replies; 18+ messages in thread
From: Kang Kai @ 2016-02-22 8:41 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 670 bytes --]
On 2016年02月18日 07:49, Burton, Ross wrote:
>
> On 17 February 2016 at 05:40, Kang Kai <Kai.Kang@windriver.com
> <mailto:Kai.Kang@windriver.com>> wrote:
>
> I prefer to keep such check. There is more than one case that
> causes operation error such as pass wrong parameters. The messages
> are helpful for developers.
>
>
> In that case the groupadd will return an error code. What situations
> with proper locking will result in groupadd running successfully but
> the group not being added?
Sorry for late reply.
I think I understand your meanings. V3 will be sent.
--Kai
>
> Ross
--
Regards,
Neil | Kai Kang
[-- Attachment #2: Type: text/html, Size: 2065 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 0/1] V3: Replace retry logic in useradd with flock
@ 2016-02-23 2:45 kai.kang
2016-02-23 2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
0 siblings, 1 reply; 18+ messages in thread
From: kai.kang @ 2016-02-23 2:45 UTC (permalink / raw)
To: ross.burton; +Cc: openembedded-core
From: Kai Kang <kai.kang@windriver.com>
V3:
* remove checks before operations such as useradd/userdel etc.
The following changes since commit ea8c34e976c11757ed9869c51b48f39050e25708:
libnewt: Fix build with PIE flags (2016-02-21 09:32:43 +0000)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib kangkai/flockv3
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/flockv3
Kai Kang (1):
useradd_base.bbclass: fix simultaneous with flock
meta/classes/useradd.bbclass | 6 +-
meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
2 files changed, 40 insertions(+), 152 deletions(-)
--
2.6.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 2:45 [PATCH 0/1] V3: Replace retry logic in useradd with flock kai.kang
@ 2016-02-23 2:45 ` kai.kang
2016-02-23 3:05 ` Khem Raj
2016-02-23 16:29 ` Burton, Ross
0 siblings, 2 replies; 18+ messages in thread
From: kai.kang @ 2016-02-23 2:45 UTC (permalink / raw)
To: ross.burton; +Cc: openembedded-core
From: Kai Kang <kai.kang@windriver.com>
When perform useradd during populate sysroot, it locks files passwd.lock
and group.lock at same time. And then it meets a dead lock issue
randomly.
Use flock to reslove it by using an universal lock file for all the
user and group related operations.
[YOCTO #9022]
Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
meta/classes/useradd.bbclass | 6 +-
meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
2 files changed, 40 insertions(+), 152 deletions(-)
diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index c960656..0a6f2be 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupadd "$SYSROOT" "$OPT $opts" 10
+ perform_groupadd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_useradd "$SYSROOT" "$OPT $opts" 10
+ perform_useradd "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
@@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
while test "x$opts" != "x"; do
- perform_groupmems "$SYSROOT" "$OPT $opts" 10
+ perform_groupmems "$SYSROOT" "$OPT $opts"
if test "x$opts" = "x$remaining"; then
break
fi
diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index ab3cd35..7c1cc13 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -4,7 +4,7 @@
# The following functions basically have similar logic.
# *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
# *) Error out if an error occurs.
# Note that before invoking these functions, make sure the global variable
@@ -12,65 +12,32 @@
perform_groupadd () {
local rootdir="$1"
- local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+ local opts=`echo $2 | sed s/\'/\"/g`
+ bbnote "${PN}: Performing groupadd with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
local groupname=`echo "$opts" | awk '{ print $NF }'`
- local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ local group_exists=`grep "^$groupname:" $rootdir/etc/group`
if test "x$group_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupadd $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" = "x"; then
- bbwarn "${PN}: groupadd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbnote "${PN}: group $groupname already exists, not re-creating it"
+ bbfatal "${PN}: groupadd command did not succeed."
fi
}
perform_useradd () {
local rootdir="$1"
- local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+ local opts=`echo $2 | sed s/\'/\"/g`
+ bbnote "${PN}: Performing useradd with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO useradd $opts\' || true
local username=`echo "$opts" | awk '{ print $NF }'`
- local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ local user_exists=`grep "^$username:" $rootdir/etc/passwd`
if test "x$user_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO useradd $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" = "x"; then
- bbwarn "${PN}: useradd command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbnote "${PN}: user $username already exists, not re-creating it"
+ bbfatal "${PN}: useradd command did not succeed."
fi
}
perform_groupmems () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupmems with [$opts]"
local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
bbnote "${PN}: Running groupmems command with group $groupname and user $username"
@@ -82,30 +49,13 @@ perform_groupmems () {
gshadow="no"
touch $rootdir${sysconfdir}/gshadow
fi
- local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
+ local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
if test "x$mem_exists" = "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmems $opts || true
- mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
- if test "x$mem_exists" = "x"; then
- bbwarn "${PN}: groupmems command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- if test "x$gshadow" = "xno"; then
- rm -f $rootdir${sysconfdir}/gshadow
- rm -f $rootdir${sysconfdir}/gshadow-
- fi
- bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbnote "${PN}: group $groupname already contains $username, not re-adding it"
+ bbfatal "${PN}: groupmems command did not succeed."
fi
+
if test "x$gshadow" = "xno"; then
rm -f $rootdir${sysconfdir}/gshadow
rm -f $rootdir${sysconfdir}/gshadow-
@@ -115,56 +65,24 @@ perform_groupmems () {
perform_groupdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing groupdel with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
local groupname=`echo "$opts" | awk '{ print $NF }'`
- local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+ local group_exists=`grep "^$groupname:" $rootdir/etc/group`
if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupdel $opts || true
- group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" != "x"; then
- bbwarn "${PN}: groupdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbnote "${PN}: group $groupname doesn't exist, not removing it"
+ bbfatal "${PN}: groupdel command did not succeed."
fi
}
perform_userdel () {
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+ bbnote "${PN}: Performing userdel with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
local username=`echo "$opts" | awk '{ print $NF }'`
- local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+ local user_exists=`grep "^$username:" $rootdir/etc/passwd`
if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO userdel $opts || true
- user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" != "x"; then
- bbwarn "${PN}: userdel command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbnote "${PN}: user $username doesn't exist, not removing it"
+ bbfatal "${PN}: userdel command did not succeed."
fi
}
@@ -172,59 +90,29 @@ perform_groupmod () {
# Other than the return value of groupmod, there's no simple way to judge whether the command
# succeeds, so we disable -e option temporarily
set +e
+
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
- local groupname=`echo "$opts" | awk '{ print $NF }'`
- local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
- if test "x$group_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO groupmod $opts
- if test $? != 0; then
- bbwarn "${PN}: groupmod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
+ bbnote "${PN}: Performing groupmod with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
+ if test $? != 0; then
+ bbwarn "${PN}: groupmod command did not succeed."
fi
+
set -e
}
perform_usermod () {
# Same reason with groupmod, temporarily disable -e option
set +e
+
local rootdir="$1"
local opts="$2"
- local retries="$3"
- bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
- local username=`echo "$opts" | awk '{ print $NF }'`
- local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
- if test "x$user_exists" != "x"; then
- local count=0
- while true; do
- eval $PSEUDO usermod $opts
- if test $? != 0; then
- bbwarn "${PN}: usermod command did not succeed. Retrying..."
- else
- break
- fi
- count=`expr $count + 1`
- if test $count = $retries; then
- bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
- fi
- sleep $count
- done
- else
- bbwarn "${PN}: user $username doesn't exist, unable to modify it"
+ bbnote "${PN}: Performing usermod with [$opts]"
+ eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
+ if test $? != 0; then
+ bbfatal "${PN}: usermod command did not succeed."
fi
+
set -e
}
--
2.6.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
@ 2016-02-23 3:05 ` Khem Raj
2016-02-23 3:35 ` Kang Kai
2016-02-23 16:29 ` Burton, Ross
1 sibling, 1 reply; 18+ messages in thread
From: Khem Raj @ 2016-02-23 3:05 UTC (permalink / raw)
To: Kang Kai; +Cc: Patches and discussions about the oe-core layer
Patch subject is unclear. Patch itself looks ok
On Mon, Feb 22, 2016 at 9:45 PM, <kai.kang@windriver.com> wrote:
> From: Kai Kang <kai.kang@windriver.com>
>
> When perform useradd during populate sysroot, it locks files passwd.lock
> and group.lock at same time. And then it meets a dead lock issue
> randomly.
>
> Use flock to reslove it by using an universal lock file for all the
> user and group related operations.
>
> [YOCTO #9022]
>
> Signed-off-by: Kai Kang <kai.kang@windriver.com>
> ---
> meta/classes/useradd.bbclass | 6 +-
> meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
> 2 files changed, 40 insertions(+), 152 deletions(-)
>
> diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
> index c960656..0a6f2be 100644
> --- a/meta/classes/useradd.bbclass
> +++ b/meta/classes/useradd.bbclass
> @@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
> opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
> remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
> while test "x$opts" != "x"; do
> - perform_groupadd "$SYSROOT" "$OPT $opts" 10
> + perform_groupadd "$SYSROOT" "$OPT $opts"
> if test "x$opts" = "x$remaining"; then
> break
> fi
> @@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
> opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
> remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
> while test "x$opts" != "x"; do
> - perform_useradd "$SYSROOT" "$OPT $opts" 10
> + perform_useradd "$SYSROOT" "$OPT $opts"
> if test "x$opts" = "x$remaining"; then
> break
> fi
> @@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
> opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
> remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
> while test "x$opts" != "x"; do
> - perform_groupmems "$SYSROOT" "$OPT $opts" 10
> + perform_groupmems "$SYSROOT" "$OPT $opts"
> if test "x$opts" = "x$remaining"; then
> break
> fi
> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
> index ab3cd35..7c1cc13 100644
> --- a/meta/classes/useradd_base.bbclass
> +++ b/meta/classes/useradd_base.bbclass
> @@ -4,7 +4,7 @@
>
> # The following functions basically have similar logic.
> # *) Perform necessary checks before invoking the actual command
> -# *) Invoke the actual command, make retries if necessary
> +# *) Invoke the actual command with flock
> # *) Error out if an error occurs.
>
> # Note that before invoking these functions, make sure the global variable
> @@ -12,65 +12,32 @@
>
> perform_groupadd () {
> local rootdir="$1"
> - local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
> + local opts=`echo $2 | sed s/\'/\"/g`
> + bbnote "${PN}: Performing groupadd with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
> local groupname=`echo "$opts" | awk '{ print $NF }'`
> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> + local group_exists=`grep "^$groupname:" $rootdir/etc/group`
> if test "x$group_exists" = "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO groupadd $opts || true
> - group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> - if test "x$group_exists" = "x"; then
> - bbwarn "${PN}: groupadd command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbnote "${PN}: group $groupname already exists, not re-creating it"
> + bbfatal "${PN}: groupadd command did not succeed."
> fi
> }
>
> perform_useradd () {
> local rootdir="$1"
> - local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
> + local opts=`echo $2 | sed s/\'/\"/g`
> + bbnote "${PN}: Performing useradd with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO useradd $opts\' || true
> local username=`echo "$opts" | awk '{ print $NF }'`
> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> + local user_exists=`grep "^$username:" $rootdir/etc/passwd`
> if test "x$user_exists" = "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO useradd $opts || true
> - user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> - if test "x$user_exists" = "x"; then
> - bbwarn "${PN}: useradd command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbnote "${PN}: user $username already exists, not re-creating it"
> + bbfatal "${PN}: useradd command did not succeed."
> fi
> }
>
> perform_groupmems () {
> local rootdir="$1"
> local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
> + bbnote "${PN}: Performing groupmems with [$opts]"
> local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
> local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
> bbnote "${PN}: Running groupmems command with group $groupname and user $username"
> @@ -82,30 +49,13 @@ perform_groupmems () {
> gshadow="no"
> touch $rootdir${sysconfdir}/gshadow
> fi
> - local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
> +
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
> + local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
> if test "x$mem_exists" = "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO groupmems $opts || true
> - mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
> - if test "x$mem_exists" = "x"; then
> - bbwarn "${PN}: groupmems command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - if test "x$gshadow" = "xno"; then
> - rm -f $rootdir${sysconfdir}/gshadow
> - rm -f $rootdir${sysconfdir}/gshadow-
> - fi
> - bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbnote "${PN}: group $groupname already contains $username, not re-adding it"
> + bbfatal "${PN}: groupmems command did not succeed."
> fi
> +
> if test "x$gshadow" = "xno"; then
> rm -f $rootdir${sysconfdir}/gshadow
> rm -f $rootdir${sysconfdir}/gshadow-
> @@ -115,56 +65,24 @@ perform_groupmems () {
> perform_groupdel () {
> local rootdir="$1"
> local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
> + bbnote "${PN}: Performing groupdel with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
> local groupname=`echo "$opts" | awk '{ print $NF }'`
> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> + local group_exists=`grep "^$groupname:" $rootdir/etc/group`
> if test "x$group_exists" != "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO groupdel $opts || true
> - group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> - if test "x$group_exists" != "x"; then
> - bbwarn "${PN}: groupdel command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbnote "${PN}: group $groupname doesn't exist, not removing it"
> + bbfatal "${PN}: groupdel command did not succeed."
> fi
> }
>
> perform_userdel () {
> local rootdir="$1"
> local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
> + bbnote "${PN}: Performing userdel with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
> local username=`echo "$opts" | awk '{ print $NF }'`
> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> + local user_exists=`grep "^$username:" $rootdir/etc/passwd`
> if test "x$user_exists" != "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO userdel $opts || true
> - user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> - if test "x$user_exists" != "x"; then
> - bbwarn "${PN}: userdel command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbnote "${PN}: user $username doesn't exist, not removing it"
> + bbfatal "${PN}: userdel command did not succeed."
> fi
> }
>
> @@ -172,59 +90,29 @@ perform_groupmod () {
> # Other than the return value of groupmod, there's no simple way to judge whether the command
> # succeeds, so we disable -e option temporarily
> set +e
> +
> local rootdir="$1"
> local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
> - local groupname=`echo "$opts" | awk '{ print $NF }'`
> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> - if test "x$group_exists" != "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO groupmod $opts
> - if test $? != 0; then
> - bbwarn "${PN}: groupmod command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
> + bbnote "${PN}: Performing groupmod with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
> + if test $? != 0; then
> + bbwarn "${PN}: groupmod command did not succeed."
> fi
> +
> set -e
> }
>
> perform_usermod () {
> # Same reason with groupmod, temporarily disable -e option
> set +e
> +
> local rootdir="$1"
> local opts="$2"
> - local retries="$3"
> - bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
> - local username=`echo "$opts" | awk '{ print $NF }'`
> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> - if test "x$user_exists" != "x"; then
> - local count=0
> - while true; do
> - eval $PSEUDO usermod $opts
> - if test $? != 0; then
> - bbwarn "${PN}: usermod command did not succeed. Retrying..."
> - else
> - break
> - fi
> - count=`expr $count + 1`
> - if test $count = $retries; then
> - bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
> - fi
> - sleep $count
> - done
> - else
> - bbwarn "${PN}: user $username doesn't exist, unable to modify it"
> + bbnote "${PN}: Performing usermod with [$opts]"
> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
> + if test $? != 0; then
> + bbfatal "${PN}: usermod command did not succeed."
> fi
> +
> set -e
> }
> --
> 2.6.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 3:05 ` Khem Raj
@ 2016-02-23 3:35 ` Kang Kai
2016-02-23 7:25 ` Richard Purdie
0 siblings, 1 reply; 18+ messages in thread
From: Kang Kai @ 2016-02-23 3:35 UTC (permalink / raw)
To: Khem Raj; +Cc: Patches and discussions about the oe-core layer
On 2016年02月23日 11:05, Khem Raj wrote:
> Patch subject is unclear. Patch itself looks ok
How about use the title of Yocto #9022:
useradd_base.bbclass: replace retry logic with flock
Thanks,
Kai
>
> On Mon, Feb 22, 2016 at 9:45 PM, <kai.kang@windriver.com> wrote:
>> From: Kai Kang <kai.kang@windriver.com>
>>
>> When perform useradd during populate sysroot, it locks files passwd.lock
>> and group.lock at same time. And then it meets a dead lock issue
>> randomly.
>>
>> Use flock to reslove it by using an universal lock file for all the
>> user and group related operations.
>>
>> [YOCTO #9022]
>>
>> Signed-off-by: Kai Kang <kai.kang@windriver.com>
>> ---
>> meta/classes/useradd.bbclass | 6 +-
>> meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
>> 2 files changed, 40 insertions(+), 152 deletions(-)
>>
>> diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
>> index c960656..0a6f2be 100644
>> --- a/meta/classes/useradd.bbclass
>> +++ b/meta/classes/useradd.bbclass
>> @@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
>> opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
>> remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
>> while test "x$opts" != "x"; do
>> - perform_groupadd "$SYSROOT" "$OPT $opts" 10
>> + perform_groupadd "$SYSROOT" "$OPT $opts"
>> if test "x$opts" = "x$remaining"; then
>> break
>> fi
>> @@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
>> opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
>> remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
>> while test "x$opts" != "x"; do
>> - perform_useradd "$SYSROOT" "$OPT $opts" 10
>> + perform_useradd "$SYSROOT" "$OPT $opts"
>> if test "x$opts" = "x$remaining"; then
>> break
>> fi
>> @@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
>> opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
>> remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
>> while test "x$opts" != "x"; do
>> - perform_groupmems "$SYSROOT" "$OPT $opts" 10
>> + perform_groupmems "$SYSROOT" "$OPT $opts"
>> if test "x$opts" = "x$remaining"; then
>> break
>> fi
>> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
>> index ab3cd35..7c1cc13 100644
>> --- a/meta/classes/useradd_base.bbclass
>> +++ b/meta/classes/useradd_base.bbclass
>> @@ -4,7 +4,7 @@
>>
>> # The following functions basically have similar logic.
>> # *) Perform necessary checks before invoking the actual command
>> -# *) Invoke the actual command, make retries if necessary
>> +# *) Invoke the actual command with flock
>> # *) Error out if an error occurs.
>>
>> # Note that before invoking these functions, make sure the global variable
>> @@ -12,65 +12,32 @@
>>
>> perform_groupadd () {
>> local rootdir="$1"
>> - local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
>> + local opts=`echo $2 | sed s/\'/\"/g`
>> + bbnote "${PN}: Performing groupadd with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
>> local groupname=`echo "$opts" | awk '{ print $NF }'`
>> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> + local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>> if test "x$group_exists" = "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO groupadd $opts || true
>> - group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> - if test "x$group_exists" = "x"; then
>> - bbwarn "${PN}: groupadd command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbnote "${PN}: group $groupname already exists, not re-creating it"
>> + bbfatal "${PN}: groupadd command did not succeed."
>> fi
>> }
>>
>> perform_useradd () {
>> local rootdir="$1"
>> - local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
>> + local opts=`echo $2 | sed s/\'/\"/g`
>> + bbnote "${PN}: Performing useradd with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO useradd $opts\' || true
>> local username=`echo "$opts" | awk '{ print $NF }'`
>> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> + local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>> if test "x$user_exists" = "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO useradd $opts || true
>> - user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> - if test "x$user_exists" = "x"; then
>> - bbwarn "${PN}: useradd command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbnote "${PN}: user $username already exists, not re-creating it"
>> + bbfatal "${PN}: useradd command did not succeed."
>> fi
>> }
>>
>> perform_groupmems () {
>> local rootdir="$1"
>> local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
>> + bbnote "${PN}: Performing groupmems with [$opts]"
>> local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
>> local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
>> bbnote "${PN}: Running groupmems command with group $groupname and user $username"
>> @@ -82,30 +49,13 @@ perform_groupmems () {
>> gshadow="no"
>> touch $rootdir${sysconfdir}/gshadow
>> fi
>> - local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
>> +
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
>> + local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
>> if test "x$mem_exists" = "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO groupmems $opts || true
>> - mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
>> - if test "x$mem_exists" = "x"; then
>> - bbwarn "${PN}: groupmems command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - if test "x$gshadow" = "xno"; then
>> - rm -f $rootdir${sysconfdir}/gshadow
>> - rm -f $rootdir${sysconfdir}/gshadow-
>> - fi
>> - bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbnote "${PN}: group $groupname already contains $username, not re-adding it"
>> + bbfatal "${PN}: groupmems command did not succeed."
>> fi
>> +
>> if test "x$gshadow" = "xno"; then
>> rm -f $rootdir${sysconfdir}/gshadow
>> rm -f $rootdir${sysconfdir}/gshadow-
>> @@ -115,56 +65,24 @@ perform_groupmems () {
>> perform_groupdel () {
>> local rootdir="$1"
>> local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
>> + bbnote "${PN}: Performing groupdel with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
>> local groupname=`echo "$opts" | awk '{ print $NF }'`
>> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> + local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>> if test "x$group_exists" != "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO groupdel $opts || true
>> - group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> - if test "x$group_exists" != "x"; then
>> - bbwarn "${PN}: groupdel command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbnote "${PN}: group $groupname doesn't exist, not removing it"
>> + bbfatal "${PN}: groupdel command did not succeed."
>> fi
>> }
>>
>> perform_userdel () {
>> local rootdir="$1"
>> local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
>> + bbnote "${PN}: Performing userdel with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
>> local username=`echo "$opts" | awk '{ print $NF }'`
>> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> + local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>> if test "x$user_exists" != "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO userdel $opts || true
>> - user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> - if test "x$user_exists" != "x"; then
>> - bbwarn "${PN}: userdel command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbnote "${PN}: user $username doesn't exist, not removing it"
>> + bbfatal "${PN}: userdel command did not succeed."
>> fi
>> }
>>
>> @@ -172,59 +90,29 @@ perform_groupmod () {
>> # Other than the return value of groupmod, there's no simple way to judge whether the command
>> # succeeds, so we disable -e option temporarily
>> set +e
>> +
>> local rootdir="$1"
>> local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
>> - local groupname=`echo "$opts" | awk '{ print $NF }'`
>> - local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> - if test "x$group_exists" != "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO groupmod $opts
>> - if test $? != 0; then
>> - bbwarn "${PN}: groupmod command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
>> + bbnote "${PN}: Performing groupmod with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
>> + if test $? != 0; then
>> + bbwarn "${PN}: groupmod command did not succeed."
>> fi
>> +
>> set -e
>> }
>>
>> perform_usermod () {
>> # Same reason with groupmod, temporarily disable -e option
>> set +e
>> +
>> local rootdir="$1"
>> local opts="$2"
>> - local retries="$3"
>> - bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
>> - local username=`echo "$opts" | awk '{ print $NF }'`
>> - local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> - if test "x$user_exists" != "x"; then
>> - local count=0
>> - while true; do
>> - eval $PSEUDO usermod $opts
>> - if test $? != 0; then
>> - bbwarn "${PN}: usermod command did not succeed. Retrying..."
>> - else
>> - break
>> - fi
>> - count=`expr $count + 1`
>> - if test $count = $retries; then
>> - bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
>> - fi
>> - sleep $count
>> - done
>> - else
>> - bbwarn "${PN}: user $username doesn't exist, unable to modify it"
>> + bbnote "${PN}: Performing usermod with [$opts]"
>> + eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
>> + if test $? != 0; then
>> + bbfatal "${PN}: usermod command did not succeed."
>> fi
>> +
>> set -e
>> }
>> --
>> 2.6.1
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
Regards,
Neil | Kai Kang
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 3:35 ` Kang Kai
@ 2016-02-23 7:25 ` Richard Purdie
2016-02-23 7:31 ` Kang Kai
0 siblings, 1 reply; 18+ messages in thread
From: Richard Purdie @ 2016-02-23 7:25 UTC (permalink / raw)
To: Kang Kai, Khem Raj; +Cc: Patches and discussions about the oe-core layer
On Tue, 2016-02-23 at 11:35 +0800, Kang Kai wrote:
> On 2016年02月23日 11:05, Khem Raj wrote:
> > Patch subject is unclear. Patch itself looks ok
>
> How about use the title of Yocto #9022:
>
> useradd_base.bbclass: replace retry logic with flock
Sounds good to me, thanks!
Richard
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 7:25 ` Richard Purdie
@ 2016-02-23 7:31 ` Kang Kai
0 siblings, 0 replies; 18+ messages in thread
From: Kang Kai @ 2016-02-23 7:31 UTC (permalink / raw)
To: Richard Purdie, Khem Raj; +Cc: Patches and discussions about the oe-core layer
On 2016年02月23日 15:25, Richard Purdie wrote:
> On Tue, 2016-02-23 at 11:35 +0800, Kang Kai wrote:
>> On 2016年02月23日 11:05, Khem Raj wrote:
>>> Patch subject is unclear. Patch itself looks ok
>> How about use the title of Yocto #9022:
>>
>> useradd_base.bbclass: replace retry logic with flock
> Sounds good to me, thanks!
Updated the git repo:
git://git.pokylinux.org/poky-contrib kangkai/flockv3
Thanks.
--Kai
>
> Richard
>
--
Regards,
Neil | Kai Kang
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
2016-02-23 3:05 ` Khem Raj
@ 2016-02-23 16:29 ` Burton, Ross
2016-02-23 17:01 ` Burton, Ross
1 sibling, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-23 16:29 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]
On 23 February 2016 at 02:45, <kai.kang@windriver.com> wrote:
> When perform useradd during populate sysroot, it locks files passwd.lock
> and group.lock at same time. And then it meets a dead lock issue
> randomly.
>
> Use flock to reslove it by using an universal lock file for all the
> user and group related operations.
>
I got an error when openssh's do_install was running useradds to the
sysroot:
| DEBUG: Executing shell function useradd_sysroot
| Running useradd commands...
|
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: useradd_sysroot (log file is located at
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/
openssh_7.1p2.bb, do_install) failed with exit code '1'
262 perform_useradd() {
263 local rootdir="$1"
264 local opts=`echo $2 | sed s/\'/\"/g`
265 bbnote "openssh: Performing useradd with [$opts]"
Ross
[-- Attachment #2: Type: text/html, Size: 2250 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 16:29 ` Burton, Ross
@ 2016-02-23 17:01 ` Burton, Ross
2016-02-23 17:10 ` Burton, Ross
0 siblings, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-23 17:01 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 1536 bytes --]
On 23 February 2016 at 16:29, Burton, Ross <ross.burton@intel.com> wrote:
> I got an error when openssh's do_install was running useradds to the
> sysroot:
>
> | DEBUG: Executing shell function useradd_sysroot
> | Running useradd commands...
> |
> /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
> 264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
> variable name
> | WARNING: exit code 2 from a shell command.
> | ERROR: Function failed: useradd_sysroot (log file is located at
> /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
> ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/
> openssh_7.1p2.bb, do_install) failed with exit code '1'
>
> 262 perform_useradd() {
> 263 local rootdir="$1"
> 264 local opts=`echo $2 | sed s/\'/\"/g`
> 265 bbnote "openssh: Performing useradd with [$opts]"
>
The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all
shell scripts to do set -x) tells me this is due to missing quotes around
the opts assignment:
+ local opts=--root /data/poky-master/tmp-glibc/sysroots/intel-core2-32
--system --no-create-home --home-dir /var/run/sshd --shell /bin/false
--user-group sshd
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.16570:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name
Ross
[-- Attachment #2: Type: text/html, Size: 2583 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 17:01 ` Burton, Ross
@ 2016-02-23 17:10 ` Burton, Ross
2016-02-24 8:51 ` Kang Kai
0 siblings, 1 reply; 18+ messages in thread
From: Burton, Ross @ 2016-02-23 17:10 UTC (permalink / raw)
To: Kang Kai; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 341 bytes --]
On 23 February 2016 at 17:01, Burton, Ross <ross.burton@intel.com> wrote:
> The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all
> shell scripts to do set -x) tells me this is due to missing quotes around
> the opts assignment:
>
Why do some of the opts assignments just do opts=$2, and others do a sed?
Ross
[-- Attachment #2: Type: text/html, Size: 744 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
2016-02-23 17:10 ` Burton, Ross
@ 2016-02-24 8:51 ` Kang Kai
0 siblings, 0 replies; 18+ messages in thread
From: Kang Kai @ 2016-02-24 8:51 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 899 bytes --]
On 2016年02月24日 01:10, Burton, Ross wrote:
>
> On 23 February 2016 at 17:01, Burton, Ross <ross.burton@intel.com
> <mailto:ross.burton@intel.com>> wrote:
>
> The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to
> get all shell scripts to do set -x) tells me this is due to
> missing quotes around the opts assignment:
>
>
> Why do some of the opts assignments just do opts=$2, and others do a sed?
In the previous version, there are:
local opts="$2"
...
opts=`echo $opts | sed s/\'/\"/g`
And I want to refactor them but lost the double quotes.
Only do sed operations for useradd and groupadd that they may be passed in "opts" with space but others didn't.
I'll restore to previous version.
Sorry for inconvenience that I built pass on Ubuntu 14.04 so didn't find
this error.
--Kai
>
> Ross
--
Regards,
Neil | Kai Kang
[-- Attachment #2: Type: text/html, Size: 2261 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2016-02-24 8:51 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-23 2:45 [PATCH 0/1] V3: Replace retry logic in useradd with flock kai.kang
2016-02-23 2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
2016-02-23 3:05 ` Khem Raj
2016-02-23 3:35 ` Kang Kai
2016-02-23 7:25 ` Richard Purdie
2016-02-23 7:31 ` Kang Kai
2016-02-23 16:29 ` Burton, Ross
2016-02-23 17:01 ` Burton, Ross
2016-02-23 17:10 ` Burton, Ross
2016-02-24 8:51 ` Kang Kai
-- strict thread matches above, loose matches on Subject: below --
2016-02-16 7:14 [PATCH 0/1] V2: Replace retry logic in useradd " kai.kang
2016-02-16 7:14 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
2016-02-16 12:09 ` Burton, Ross
2016-02-17 5:40 ` Kang Kai
2016-02-17 23:49 ` Burton, Ross
2016-02-22 8:41 ` Kang Kai
2016-02-15 1:59 [PATCH 0/1] Replace retry logic in useradd " kai.kang
2016-02-15 1:59 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
2016-02-15 12:31 ` Burton, Ross
2016-02-16 2:14 ` Kang Kai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox