Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH 0/1] V3: Replace retry logic in useradd with flock
@ 2016-02-23  2:45 kai.kang
  2016-02-23  2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
  0 siblings, 1 reply; 10+ messages in thread
From: kai.kang @ 2016-02-23  2:45 UTC (permalink / raw)
  To: ross.burton; +Cc: openembedded-core

From: Kai Kang <kai.kang@windriver.com>

V3:

* remove checks before operations such as useradd/userdel etc.

The following changes since commit ea8c34e976c11757ed9869c51b48f39050e25708:

  libnewt: Fix build with PIE flags (2016-02-21 09:32:43 +0000)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib kangkai/flockv3
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/flockv3

Kai Kang (1):
  useradd_base.bbclass: fix simultaneous with flock

 meta/classes/useradd.bbclass      |   6 +-
 meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
 2 files changed, 40 insertions(+), 152 deletions(-)

-- 
2.6.1



^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  2:45 [PATCH 0/1] V3: Replace retry logic in useradd with flock kai.kang
@ 2016-02-23  2:45 ` kai.kang
  2016-02-23  3:05   ` Khem Raj
  2016-02-23 16:29   ` Burton, Ross
  0 siblings, 2 replies; 10+ messages in thread
From: kai.kang @ 2016-02-23  2:45 UTC (permalink / raw)
  To: ross.burton; +Cc: openembedded-core

From: Kai Kang <kai.kang@windriver.com>

When perform useradd during populate sysroot, it locks files passwd.lock
and group.lock at same time. And then it meets a dead lock issue
randomly.

Use flock to reslove it by using an universal lock file for all the
user and group related operations.

[YOCTO #9022]

Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
 meta/classes/useradd.bbclass      |   6 +-
 meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
 2 files changed, 40 insertions(+), 152 deletions(-)

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index c960656..0a6f2be 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_groupadd "$SYSROOT" "$OPT $opts" 10
+		perform_groupadd "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
@@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_useradd "$SYSROOT" "$OPT $opts" 10
+		perform_useradd "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
@@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_groupmems "$SYSROOT" "$OPT $opts" 10
+		perform_groupmems "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index ab3cd35..7c1cc13 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -4,7 +4,7 @@
 
 # The following functions basically have similar logic.
 # *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
 # *) Error out if an error occurs.
 
 # Note that before invoking these functions, make sure the global variable
@@ -12,65 +12,32 @@
 
 perform_groupadd () {
 	local rootdir="$1"
-	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+	local opts=`echo $2 | sed s/\'/\"/g`
+	bbnote "${PN}: Performing groupadd with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
 	local groupname=`echo "$opts" | awk '{ print $NF }'`
-	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+	local group_exists=`grep "^$groupname:" $rootdir/etc/group`
 	if test "x$group_exists" = "x"; then
-		local count=0
-		while true; do
-			eval $PSEUDO groupadd $opts || true
-			group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-			if test "x$group_exists" = "x"; then
-				bbwarn "${PN}: groupadd command did not succeed. Retrying..."
-			else
-				break
-			fi
-			count=`expr $count + 1`
-			if test $count = $retries; then
-				bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
-			fi
-                        sleep $count
-		done
-	else
-		bbnote "${PN}: group $groupname already exists, not re-creating it"
+		bbfatal "${PN}: groupadd command did not succeed."
 	fi
 }
 
 perform_useradd () {
 	local rootdir="$1"
-	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+	local opts=`echo $2 | sed s/\'/\"/g`
+	bbnote "${PN}: Performing useradd with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c  \'$PSEUDO useradd $opts\' || true
 	local username=`echo "$opts" | awk '{ print $NF }'`
-	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+	local user_exists=`grep "^$username:" $rootdir/etc/passwd`
 	if test "x$user_exists" = "x"; then
-	       local count=0
-	       while true; do
-		       eval $PSEUDO useradd $opts || true
-		       user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-		       if test "x$user_exists" = "x"; then
-			       bbwarn "${PN}: useradd command did not succeed. Retrying..."
-		       else
-			       break
-		       fi
-		       count=`expr $count + 1`
-		       if test $count = $retries; then
-				bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
-		       fi
-		       sleep $count
-	       done
-	else
-		bbnote "${PN}: user $username already exists, not re-creating it"
+		bbfatal "${PN}: useradd command did not succeed."
 	fi
 }
 
 perform_groupmems () {
 	local rootdir="$1"
 	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+	bbnote "${PN}: Performing groupmems with [$opts]"
 	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
 	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
 	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
@@ -82,30 +49,13 @@ perform_groupmems () {
 		gshadow="no"
 		touch $rootdir${sysconfdir}/gshadow
 	fi
-	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
+	local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
 	if test "x$mem_exists" = "x"; then
-		local count=0
-		while true; do
-			eval $PSEUDO groupmems $opts || true
-			mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
-			if test "x$mem_exists" = "x"; then
-				bbwarn "${PN}: groupmems command did not succeed. Retrying..."
-			else
-				break
-			fi
-			count=`expr $count + 1`
-			if test $count = $retries; then
-				if test "x$gshadow" = "xno"; then
-					rm -f $rootdir${sysconfdir}/gshadow
-					rm -f $rootdir${sysconfdir}/gshadow-
-				fi
-				bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
-			fi
-			sleep $count
-		done
-	else
-		bbnote "${PN}: group $groupname already contains $username, not re-adding it"
+		bbfatal "${PN}: groupmems command did not succeed."
 	fi
+
 	if test "x$gshadow" = "xno"; then
 		rm -f $rootdir${sysconfdir}/gshadow
 		rm -f $rootdir${sysconfdir}/gshadow-
@@ -115,56 +65,24 @@ perform_groupmems () {
 perform_groupdel () {
 	local rootdir="$1"
 	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+	bbnote "${PN}: Performing groupdel with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
 	local groupname=`echo "$opts" | awk '{ print $NF }'`
-	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+	local group_exists=`grep "^$groupname:" $rootdir/etc/group`
 	if test "x$group_exists" != "x"; then
-		local count=0
-		while true; do
-			eval $PSEUDO groupdel $opts || true
-			group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-			if test "x$group_exists" != "x"; then
-				bbwarn "${PN}: groupdel command did not succeed. Retrying..."
-			else
-				break
-			fi
-			count=`expr $count + 1`
-			if test $count = $retries; then
-				bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
-			fi
-			sleep $count
-		done
-	else
-		bbnote "${PN}: group $groupname doesn't exist, not removing it"
+		bbfatal "${PN}: groupdel command did not succeed."
 	fi
 }
 
 perform_userdel () {
 	local rootdir="$1"
 	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+	bbnote "${PN}: Performing userdel with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
 	local username=`echo "$opts" | awk '{ print $NF }'`
-	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+	local user_exists=`grep "^$username:" $rootdir/etc/passwd`
 	if test "x$user_exists" != "x"; then
-	       local count=0
-	       while true; do
-		       eval $PSEUDO userdel $opts || true
-		       user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-		       if test "x$user_exists" != "x"; then
-			       bbwarn "${PN}: userdel command did not succeed. Retrying..."
-		       else
-			       break
-		       fi
-		       count=`expr $count + 1`
-		       if test $count = $retries; then
-				bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
-		       fi
-		       sleep $count
-	       done
-	else
-		bbnote "${PN}: user $username doesn't exist, not removing it"
+		bbfatal "${PN}: userdel command did not succeed."
 	fi
 }
 
@@ -172,59 +90,29 @@ perform_groupmod () {
 	# Other than the return value of groupmod, there's no simple way to judge whether the command
 	# succeeds, so we disable -e option temporarily
 	set +e
+
 	local rootdir="$1"
 	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
-	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-	if test "x$group_exists" != "x"; then
-		local count=0
-		while true; do
-			eval $PSEUDO groupmod $opts
-			if test $? != 0; then
-				bbwarn "${PN}: groupmod command did not succeed. Retrying..."
-			else
-				break
-			fi
-			count=`expr $count + 1`
-			if test $count = $retries; then
-				bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
-			fi
-			sleep $count
-		done
-	else
-		bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
+	bbnote "${PN}: Performing groupmod with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
+	if test $? != 0; then
+		bbwarn "${PN}: groupmod command did not succeed."
 	fi
+
 	set -e
 }
 
 perform_usermod () {
 	# Same reason with groupmod, temporarily disable -e option
 	set +e
+
 	local rootdir="$1"
 	local opts="$2"
-	local retries="$3"
-	bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
-	local username=`echo "$opts" | awk '{ print $NF }'`
-	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-	if test "x$user_exists" != "x"; then
-	       local count=0
-	       while true; do
-		       eval $PSEUDO usermod $opts
-		       if test $? != 0; then
-			       bbwarn "${PN}: usermod command did not succeed. Retrying..."
-		       else
-			       break
-		       fi
-		       count=`expr $count + 1`
-		       if test $count = $retries; then
-				bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
-		       fi
-		       sleep $count
-	       done
-	else
-		bbwarn "${PN}: user $username doesn't exist, unable to modify it"
+	bbnote "${PN}: Performing usermod with [$opts]"
+	eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
+	if test $? != 0; then
+		bbfatal "${PN}: usermod command did not succeed."
 	fi
+
 	set -e
 }
-- 
2.6.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
@ 2016-02-23  3:05   ` Khem Raj
  2016-02-23  3:35     ` Kang Kai
  2016-02-23 16:29   ` Burton, Ross
  1 sibling, 1 reply; 10+ messages in thread
From: Khem Raj @ 2016-02-23  3:05 UTC (permalink / raw)
  To: Kang Kai; +Cc: Patches and discussions about the oe-core layer

Patch subject is unclear. Patch itself looks ok

On Mon, Feb 22, 2016 at 9:45 PM,  <kai.kang@windriver.com> wrote:
> From: Kai Kang <kai.kang@windriver.com>
>
> When perform useradd during populate sysroot, it locks files passwd.lock
> and group.lock at same time. And then it meets a dead lock issue
> randomly.
>
> Use flock to reslove it by using an universal lock file for all the
> user and group related operations.
>
> [YOCTO #9022]
>
> Signed-off-by: Kai Kang <kai.kang@windriver.com>
> ---
>  meta/classes/useradd.bbclass      |   6 +-
>  meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
>  2 files changed, 40 insertions(+), 152 deletions(-)
>
> diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
> index c960656..0a6f2be 100644
> --- a/meta/classes/useradd.bbclass
> +++ b/meta/classes/useradd.bbclass
> @@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
>         opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
>         remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
>         while test "x$opts" != "x"; do
> -               perform_groupadd "$SYSROOT" "$OPT $opts" 10
> +               perform_groupadd "$SYSROOT" "$OPT $opts"
>                 if test "x$opts" = "x$remaining"; then
>                         break
>                 fi
> @@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
>         opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
>         remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
>         while test "x$opts" != "x"; do
> -               perform_useradd "$SYSROOT" "$OPT $opts" 10
> +               perform_useradd "$SYSROOT" "$OPT $opts"
>                 if test "x$opts" = "x$remaining"; then
>                         break
>                 fi
> @@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
>         opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
>         remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
>         while test "x$opts" != "x"; do
> -               perform_groupmems "$SYSROOT" "$OPT $opts" 10
> +               perform_groupmems "$SYSROOT" "$OPT $opts"
>                 if test "x$opts" = "x$remaining"; then
>                         break
>                 fi
> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
> index ab3cd35..7c1cc13 100644
> --- a/meta/classes/useradd_base.bbclass
> +++ b/meta/classes/useradd_base.bbclass
> @@ -4,7 +4,7 @@
>
>  # The following functions basically have similar logic.
>  # *) Perform necessary checks before invoking the actual command
> -# *) Invoke the actual command, make retries if necessary
> +# *) Invoke the actual command with flock
>  # *) Error out if an error occurs.
>
>  # Note that before invoking these functions, make sure the global variable
> @@ -12,65 +12,32 @@
>
>  perform_groupadd () {
>         local rootdir="$1"
> -       local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
> +       local opts=`echo $2 | sed s/\'/\"/g`
> +       bbnote "${PN}: Performing groupadd with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
>         local groupname=`echo "$opts" | awk '{ print $NF }'`
> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> +       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>         if test "x$group_exists" = "x"; then
> -               local count=0
> -               while true; do
> -                       eval $PSEUDO groupadd $opts || true
> -                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> -                       if test "x$group_exists" = "x"; then
> -                               bbwarn "${PN}: groupadd command did not succeed. Retrying..."
> -                       else
> -                               break
> -                       fi
> -                       count=`expr $count + 1`
> -                       if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
> -                       fi
> -                        sleep $count
> -               done
> -       else
> -               bbnote "${PN}: group $groupname already exists, not re-creating it"
> +               bbfatal "${PN}: groupadd command did not succeed."
>         fi
>  }
>
>  perform_useradd () {
>         local rootdir="$1"
> -       local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
> +       local opts=`echo $2 | sed s/\'/\"/g`
> +       bbnote "${PN}: Performing useradd with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c  \'$PSEUDO useradd $opts\' || true
>         local username=`echo "$opts" | awk '{ print $NF }'`
> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> +       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>         if test "x$user_exists" = "x"; then
> -              local count=0
> -              while true; do
> -                      eval $PSEUDO useradd $opts || true
> -                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> -                      if test "x$user_exists" = "x"; then
> -                              bbwarn "${PN}: useradd command did not succeed. Retrying..."
> -                      else
> -                              break
> -                      fi
> -                      count=`expr $count + 1`
> -                      if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
> -                      fi
> -                      sleep $count
> -              done
> -       else
> -               bbnote "${PN}: user $username already exists, not re-creating it"
> +               bbfatal "${PN}: useradd command did not succeed."
>         fi
>  }
>
>  perform_groupmems () {
>         local rootdir="$1"
>         local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
> +       bbnote "${PN}: Performing groupmems with [$opts]"
>         local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
>         local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
>         bbnote "${PN}: Running groupmems command with group $groupname and user $username"
> @@ -82,30 +49,13 @@ perform_groupmems () {
>                 gshadow="no"
>                 touch $rootdir${sysconfdir}/gshadow
>         fi
> -       local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
> +
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
> +       local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
>         if test "x$mem_exists" = "x"; then
> -               local count=0
> -               while true; do
> -                       eval $PSEUDO groupmems $opts || true
> -                       mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
> -                       if test "x$mem_exists" = "x"; then
> -                               bbwarn "${PN}: groupmems command did not succeed. Retrying..."
> -                       else
> -                               break
> -                       fi
> -                       count=`expr $count + 1`
> -                       if test $count = $retries; then
> -                               if test "x$gshadow" = "xno"; then
> -                                       rm -f $rootdir${sysconfdir}/gshadow
> -                                       rm -f $rootdir${sysconfdir}/gshadow-
> -                               fi
> -                               bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
> -                       fi
> -                       sleep $count
> -               done
> -       else
> -               bbnote "${PN}: group $groupname already contains $username, not re-adding it"
> +               bbfatal "${PN}: groupmems command did not succeed."
>         fi
> +
>         if test "x$gshadow" = "xno"; then
>                 rm -f $rootdir${sysconfdir}/gshadow
>                 rm -f $rootdir${sysconfdir}/gshadow-
> @@ -115,56 +65,24 @@ perform_groupmems () {
>  perform_groupdel () {
>         local rootdir="$1"
>         local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
> +       bbnote "${PN}: Performing groupdel with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
>         local groupname=`echo "$opts" | awk '{ print $NF }'`
> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> +       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>         if test "x$group_exists" != "x"; then
> -               local count=0
> -               while true; do
> -                       eval $PSEUDO groupdel $opts || true
> -                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> -                       if test "x$group_exists" != "x"; then
> -                               bbwarn "${PN}: groupdel command did not succeed. Retrying..."
> -                       else
> -                               break
> -                       fi
> -                       count=`expr $count + 1`
> -                       if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
> -                       fi
> -                       sleep $count
> -               done
> -       else
> -               bbnote "${PN}: group $groupname doesn't exist, not removing it"
> +               bbfatal "${PN}: groupdel command did not succeed."
>         fi
>  }
>
>  perform_userdel () {
>         local rootdir="$1"
>         local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
> +       bbnote "${PN}: Performing userdel with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
>         local username=`echo "$opts" | awk '{ print $NF }'`
> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> +       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>         if test "x$user_exists" != "x"; then
> -              local count=0
> -              while true; do
> -                      eval $PSEUDO userdel $opts || true
> -                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> -                      if test "x$user_exists" != "x"; then
> -                              bbwarn "${PN}: userdel command did not succeed. Retrying..."
> -                      else
> -                              break
> -                      fi
> -                      count=`expr $count + 1`
> -                      if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
> -                      fi
> -                      sleep $count
> -              done
> -       else
> -               bbnote "${PN}: user $username doesn't exist, not removing it"
> +               bbfatal "${PN}: userdel command did not succeed."
>         fi
>  }
>
> @@ -172,59 +90,29 @@ perform_groupmod () {
>         # Other than the return value of groupmod, there's no simple way to judge whether the command
>         # succeeds, so we disable -e option temporarily
>         set +e
> +
>         local rootdir="$1"
>         local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
> -       local groupname=`echo "$opts" | awk '{ print $NF }'`
> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> -       if test "x$group_exists" != "x"; then
> -               local count=0
> -               while true; do
> -                       eval $PSEUDO groupmod $opts
> -                       if test $? != 0; then
> -                               bbwarn "${PN}: groupmod command did not succeed. Retrying..."
> -                       else
> -                               break
> -                       fi
> -                       count=`expr $count + 1`
> -                       if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
> -                       fi
> -                       sleep $count
> -               done
> -       else
> -               bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
> +       bbnote "${PN}: Performing groupmod with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
> +       if test $? != 0; then
> +               bbwarn "${PN}: groupmod command did not succeed."
>         fi
> +
>         set -e
>  }
>
>  perform_usermod () {
>         # Same reason with groupmod, temporarily disable -e option
>         set +e
> +
>         local rootdir="$1"
>         local opts="$2"
> -       local retries="$3"
> -       bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
> -       local username=`echo "$opts" | awk '{ print $NF }'`
> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
> -       if test "x$user_exists" != "x"; then
> -              local count=0
> -              while true; do
> -                      eval $PSEUDO usermod $opts
> -                      if test $? != 0; then
> -                              bbwarn "${PN}: usermod command did not succeed. Retrying..."
> -                      else
> -                              break
> -                      fi
> -                      count=`expr $count + 1`
> -                      if test $count = $retries; then
> -                               bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
> -                      fi
> -                      sleep $count
> -              done
> -       else
> -               bbwarn "${PN}: user $username doesn't exist, unable to modify it"
> +       bbnote "${PN}: Performing usermod with [$opts]"
> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
> +       if test $? != 0; then
> +               bbfatal "${PN}: usermod command did not succeed."
>         fi
> +
>         set -e
>  }
> --
> 2.6.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  3:05   ` Khem Raj
@ 2016-02-23  3:35     ` Kang Kai
  2016-02-23  7:25       ` Richard Purdie
  0 siblings, 1 reply; 10+ messages in thread
From: Kang Kai @ 2016-02-23  3:35 UTC (permalink / raw)
  To: Khem Raj; +Cc: Patches and discussions about the oe-core layer

On 2016年02月23日 11:05, Khem Raj wrote:
> Patch subject is unclear. Patch itself looks ok

How about use the  title of Yocto #9022:

useradd_base.bbclass: replace retry logic with flock


Thanks,
Kai



>
> On Mon, Feb 22, 2016 at 9:45 PM,  <kai.kang@windriver.com> wrote:
>> From: Kai Kang <kai.kang@windriver.com>
>>
>> When perform useradd during populate sysroot, it locks files passwd.lock
>> and group.lock at same time. And then it meets a dead lock issue
>> randomly.
>>
>> Use flock to reslove it by using an universal lock file for all the
>> user and group related operations.
>>
>> [YOCTO #9022]
>>
>> Signed-off-by: Kai Kang <kai.kang@windriver.com>
>> ---
>>   meta/classes/useradd.bbclass      |   6 +-
>>   meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
>>   2 files changed, 40 insertions(+), 152 deletions(-)
>>
>> diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
>> index c960656..0a6f2be 100644
>> --- a/meta/classes/useradd.bbclass
>> +++ b/meta/classes/useradd.bbclass
>> @@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
>>          opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
>>          remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
>>          while test "x$opts" != "x"; do
>> -               perform_groupadd "$SYSROOT" "$OPT $opts" 10
>> +               perform_groupadd "$SYSROOT" "$OPT $opts"
>>                  if test "x$opts" = "x$remaining"; then
>>                          break
>>                  fi
>> @@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
>>          opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
>>          remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
>>          while test "x$opts" != "x"; do
>> -               perform_useradd "$SYSROOT" "$OPT $opts" 10
>> +               perform_useradd "$SYSROOT" "$OPT $opts"
>>                  if test "x$opts" = "x$remaining"; then
>>                          break
>>                  fi
>> @@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
>>          opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
>>          remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
>>          while test "x$opts" != "x"; do
>> -               perform_groupmems "$SYSROOT" "$OPT $opts" 10
>> +               perform_groupmems "$SYSROOT" "$OPT $opts"
>>                  if test "x$opts" = "x$remaining"; then
>>                          break
>>                  fi
>> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
>> index ab3cd35..7c1cc13 100644
>> --- a/meta/classes/useradd_base.bbclass
>> +++ b/meta/classes/useradd_base.bbclass
>> @@ -4,7 +4,7 @@
>>
>>   # The following functions basically have similar logic.
>>   # *) Perform necessary checks before invoking the actual command
>> -# *) Invoke the actual command, make retries if necessary
>> +# *) Invoke the actual command with flock
>>   # *) Error out if an error occurs.
>>
>>   # Note that before invoking these functions, make sure the global variable
>> @@ -12,65 +12,32 @@
>>
>>   perform_groupadd () {
>>          local rootdir="$1"
>> -       local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
>> +       local opts=`echo $2 | sed s/\'/\"/g`
>> +       bbnote "${PN}: Performing groupadd with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
>>          local groupname=`echo "$opts" | awk '{ print $NF }'`
>> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> +       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>>          if test "x$group_exists" = "x"; then
>> -               local count=0
>> -               while true; do
>> -                       eval $PSEUDO groupadd $opts || true
>> -                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> -                       if test "x$group_exists" = "x"; then
>> -                               bbwarn "${PN}: groupadd command did not succeed. Retrying..."
>> -                       else
>> -                               break
>> -                       fi
>> -                       count=`expr $count + 1`
>> -                       if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
>> -                       fi
>> -                        sleep $count
>> -               done
>> -       else
>> -               bbnote "${PN}: group $groupname already exists, not re-creating it"
>> +               bbfatal "${PN}: groupadd command did not succeed."
>>          fi
>>   }
>>
>>   perform_useradd () {
>>          local rootdir="$1"
>> -       local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
>> +       local opts=`echo $2 | sed s/\'/\"/g`
>> +       bbnote "${PN}: Performing useradd with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c  \'$PSEUDO useradd $opts\' || true
>>          local username=`echo "$opts" | awk '{ print $NF }'`
>> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> +       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>>          if test "x$user_exists" = "x"; then
>> -              local count=0
>> -              while true; do
>> -                      eval $PSEUDO useradd $opts || true
>> -                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> -                      if test "x$user_exists" = "x"; then
>> -                              bbwarn "${PN}: useradd command did not succeed. Retrying..."
>> -                      else
>> -                              break
>> -                      fi
>> -                      count=`expr $count + 1`
>> -                      if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
>> -                      fi
>> -                      sleep $count
>> -              done
>> -       else
>> -               bbnote "${PN}: user $username already exists, not re-creating it"
>> +               bbfatal "${PN}: useradd command did not succeed."
>>          fi
>>   }
>>
>>   perform_groupmems () {
>>          local rootdir="$1"
>>          local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
>> +       bbnote "${PN}: Performing groupmems with [$opts]"
>>          local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
>>          local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
>>          bbnote "${PN}: Running groupmems command with group $groupname and user $username"
>> @@ -82,30 +49,13 @@ perform_groupmems () {
>>                  gshadow="no"
>>                  touch $rootdir${sysconfdir}/gshadow
>>          fi
>> -       local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
>> +
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
>> +       local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
>>          if test "x$mem_exists" = "x"; then
>> -               local count=0
>> -               while true; do
>> -                       eval $PSEUDO groupmems $opts || true
>> -                       mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
>> -                       if test "x$mem_exists" = "x"; then
>> -                               bbwarn "${PN}: groupmems command did not succeed. Retrying..."
>> -                       else
>> -                               break
>> -                       fi
>> -                       count=`expr $count + 1`
>> -                       if test $count = $retries; then
>> -                               if test "x$gshadow" = "xno"; then
>> -                                       rm -f $rootdir${sysconfdir}/gshadow
>> -                                       rm -f $rootdir${sysconfdir}/gshadow-
>> -                               fi
>> -                               bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
>> -                       fi
>> -                       sleep $count
>> -               done
>> -       else
>> -               bbnote "${PN}: group $groupname already contains $username, not re-adding it"
>> +               bbfatal "${PN}: groupmems command did not succeed."
>>          fi
>> +
>>          if test "x$gshadow" = "xno"; then
>>                  rm -f $rootdir${sysconfdir}/gshadow
>>                  rm -f $rootdir${sysconfdir}/gshadow-
>> @@ -115,56 +65,24 @@ perform_groupmems () {
>>   perform_groupdel () {
>>          local rootdir="$1"
>>          local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
>> +       bbnote "${PN}: Performing groupdel with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
>>          local groupname=`echo "$opts" | awk '{ print $NF }'`
>> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> +       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
>>          if test "x$group_exists" != "x"; then
>> -               local count=0
>> -               while true; do
>> -                       eval $PSEUDO groupdel $opts || true
>> -                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> -                       if test "x$group_exists" != "x"; then
>> -                               bbwarn "${PN}: groupdel command did not succeed. Retrying..."
>> -                       else
>> -                               break
>> -                       fi
>> -                       count=`expr $count + 1`
>> -                       if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
>> -                       fi
>> -                       sleep $count
>> -               done
>> -       else
>> -               bbnote "${PN}: group $groupname doesn't exist, not removing it"
>> +               bbfatal "${PN}: groupdel command did not succeed."
>>          fi
>>   }
>>
>>   perform_userdel () {
>>          local rootdir="$1"
>>          local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
>> +       bbnote "${PN}: Performing userdel with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
>>          local username=`echo "$opts" | awk '{ print $NF }'`
>> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> +       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
>>          if test "x$user_exists" != "x"; then
>> -              local count=0
>> -              while true; do
>> -                      eval $PSEUDO userdel $opts || true
>> -                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> -                      if test "x$user_exists" != "x"; then
>> -                              bbwarn "${PN}: userdel command did not succeed. Retrying..."
>> -                      else
>> -                              break
>> -                      fi
>> -                      count=`expr $count + 1`
>> -                      if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
>> -                      fi
>> -                      sleep $count
>> -              done
>> -       else
>> -               bbnote "${PN}: user $username doesn't exist, not removing it"
>> +               bbfatal "${PN}: userdel command did not succeed."
>>          fi
>>   }
>>
>> @@ -172,59 +90,29 @@ perform_groupmod () {
>>          # Other than the return value of groupmod, there's no simple way to judge whether the command
>>          # succeeds, so we disable -e option temporarily
>>          set +e
>> +
>>          local rootdir="$1"
>>          local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
>> -       local groupname=`echo "$opts" | awk '{ print $NF }'`
>> -       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>> -       if test "x$group_exists" != "x"; then
>> -               local count=0
>> -               while true; do
>> -                       eval $PSEUDO groupmod $opts
>> -                       if test $? != 0; then
>> -                               bbwarn "${PN}: groupmod command did not succeed. Retrying..."
>> -                       else
>> -                               break
>> -                       fi
>> -                       count=`expr $count + 1`
>> -                       if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
>> -                       fi
>> -                       sleep $count
>> -               done
>> -       else
>> -               bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
>> +       bbnote "${PN}: Performing groupmod with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
>> +       if test $? != 0; then
>> +               bbwarn "${PN}: groupmod command did not succeed."
>>          fi
>> +
>>          set -e
>>   }
>>
>>   perform_usermod () {
>>          # Same reason with groupmod, temporarily disable -e option
>>          set +e
>> +
>>          local rootdir="$1"
>>          local opts="$2"
>> -       local retries="$3"
>> -       bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
>> -       local username=`echo "$opts" | awk '{ print $NF }'`
>> -       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>> -       if test "x$user_exists" != "x"; then
>> -              local count=0
>> -              while true; do
>> -                      eval $PSEUDO usermod $opts
>> -                      if test $? != 0; then
>> -                              bbwarn "${PN}: usermod command did not succeed. Retrying..."
>> -                      else
>> -                              break
>> -                      fi
>> -                      count=`expr $count + 1`
>> -                      if test $count = $retries; then
>> -                               bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
>> -                      fi
>> -                      sleep $count
>> -              done
>> -       else
>> -               bbwarn "${PN}: user $username doesn't exist, unable to modify it"
>> +       bbnote "${PN}: Performing usermod with [$opts]"
>> +       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
>> +       if test $? != 0; then
>> +               bbfatal "${PN}: usermod command did not succeed."
>>          fi
>> +
>>          set -e
>>   }
>> --
>> 2.6.1
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core


-- 
Regards,
Neil | Kai Kang



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  3:35     ` Kang Kai
@ 2016-02-23  7:25       ` Richard Purdie
  2016-02-23  7:31         ` Kang Kai
  0 siblings, 1 reply; 10+ messages in thread
From: Richard Purdie @ 2016-02-23  7:25 UTC (permalink / raw)
  To: Kang Kai, Khem Raj; +Cc: Patches and discussions about the oe-core layer

On Tue, 2016-02-23 at 11:35 +0800, Kang Kai wrote:
> On 2016年02月23日 11:05, Khem Raj wrote:
> > Patch subject is unclear. Patch itself looks ok
> 
> How about use the  title of Yocto #9022:
> 
> useradd_base.bbclass: replace retry logic with flock

Sounds good to me, thanks!

Richard


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  7:25       ` Richard Purdie
@ 2016-02-23  7:31         ` Kang Kai
  0 siblings, 0 replies; 10+ messages in thread
From: Kang Kai @ 2016-02-23  7:31 UTC (permalink / raw)
  To: Richard Purdie, Khem Raj; +Cc: Patches and discussions about the oe-core layer

On 2016年02月23日 15:25, Richard Purdie wrote:
> On Tue, 2016-02-23 at 11:35 +0800, Kang Kai wrote:
>> On 2016年02月23日 11:05, Khem Raj wrote:
>>> Patch subject is unclear. Patch itself looks ok
>> How about use the  title of Yocto #9022:
>>
>> useradd_base.bbclass: replace retry logic with flock
> Sounds good to me, thanks!

Updated the git repo:

git://git.pokylinux.org/poky-contrib kangkai/flockv3


Thanks.

--Kai
>
> Richard
>


-- 
Regards,
Neil | Kai Kang



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23  2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
  2016-02-23  3:05   ` Khem Raj
@ 2016-02-23 16:29   ` Burton, Ross
  2016-02-23 17:01     ` Burton, Ross
  1 sibling, 1 reply; 10+ messages in thread
From: Burton, Ross @ 2016-02-23 16:29 UTC (permalink / raw)
  To: Kang Kai; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On 23 February 2016 at 02:45, <kai.kang@windriver.com> wrote:

> When perform useradd during populate sysroot, it locks files passwd.lock
> and group.lock at same time. And then it meets a dead lock issue
> randomly.
>
> Use flock to reslove it by using an universal lock file for all the
> user and group related operations.
>

I got an error when openssh's do_install was running useradds to the
sysroot:

| DEBUG: Executing shell function useradd_sysroot
| Running useradd commands...
|
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: useradd_sysroot (log file is located at
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/
openssh_7.1p2.bb, do_install) failed with exit code '1'

    262 perform_useradd() {
    263         local rootdir="$1"
    264         local opts=`echo $2 | sed s/\'/\"/g`
    265         bbnote "openssh: Performing useradd with [$opts]"

Ross

[-- Attachment #2: Type: text/html, Size: 2250 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23 16:29   ` Burton, Ross
@ 2016-02-23 17:01     ` Burton, Ross
  2016-02-23 17:10       ` Burton, Ross
  0 siblings, 1 reply; 10+ messages in thread
From: Burton, Ross @ 2016-02-23 17:01 UTC (permalink / raw)
  To: Kang Kai; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 1536 bytes --]

On 23 February 2016 at 16:29, Burton, Ross <ross.burton@intel.com> wrote:

> I got an error when openssh's do_install was running useradds to the
> sysroot:
>
> | DEBUG: Executing shell function useradd_sysroot
> | Running useradd commands...
> |
> /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
> 264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
> variable name
> | WARNING: exit code 2 from a shell command.
> | ERROR: Function failed: useradd_sysroot (log file is located at
> /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
> ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/
> openssh_7.1p2.bb, do_install) failed with exit code '1'
>
>     262 perform_useradd() {
>     263         local rootdir="$1"
>     264         local opts=`echo $2 | sed s/\'/\"/g`
>     265         bbnote "openssh: Performing useradd with [$opts]"
>

The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all
shell scripts to do set -x) tells me this is due to missing quotes around
the opts assignment:

+ local opts=--root /data/poky-master/tmp-glibc/sysroots/intel-core2-32
--system --no-create-home --home-dir /var/run/sshd --shell /bin/false
--user-group sshd
/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.16570:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name

Ross

[-- Attachment #2: Type: text/html, Size: 2583 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23 17:01     ` Burton, Ross
@ 2016-02-23 17:10       ` Burton, Ross
  2016-02-24  8:51         ` Kang Kai
  0 siblings, 1 reply; 10+ messages in thread
From: Burton, Ross @ 2016-02-23 17:10 UTC (permalink / raw)
  To: Kang Kai; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 341 bytes --]

On 23 February 2016 at 17:01, Burton, Ross <ross.burton@intel.com> wrote:

> The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all
> shell scripts to do set -x) tells me this is due to missing quotes around
> the opts assignment:
>

Why do some of the opts assignments just do opts=$2, and others do a sed?

Ross

[-- Attachment #2: Type: text/html, Size: 744 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock
  2016-02-23 17:10       ` Burton, Ross
@ 2016-02-24  8:51         ` Kang Kai
  0 siblings, 0 replies; 10+ messages in thread
From: Kang Kai @ 2016-02-24  8:51 UTC (permalink / raw)
  To: Burton, Ross; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On 2016年02月24日 01:10, Burton, Ross wrote:
>
> On 23 February 2016 at 17:01, Burton, Ross <ross.burton@intel.com 
> <mailto:ross.burton@intel.com>> wrote:
>
>     The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to
>     get all shell scripts to do set -x) tells me this is due to
>     missing quotes around the opts assignment:
>
>
> Why do some of the opts assignments just do opts=$2, and others do a sed?

In the previous version, there are:

local opts="$2"
...

opts=`echo $opts | sed s/\'/\"/g`

And I want to refactor them but lost the double quotes.

Only do sed operations for useradd and groupadd that they may be passed in "opts" with space but others didn't.

I'll restore to previous version.

Sorry for inconvenience that I built pass on Ubuntu 14.04 so didn't find 
this error.

--Kai

>
> Ross


-- 
Regards,
Neil | Kai Kang


[-- Attachment #2: Type: text/html, Size: 2261 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2016-02-24  8:51 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-23  2:45 [PATCH 0/1] V3: Replace retry logic in useradd with flock kai.kang
2016-02-23  2:45 ` [PATCH 1/1] useradd_base.bbclass: fix simultaneous " kai.kang
2016-02-23  3:05   ` Khem Raj
2016-02-23  3:35     ` Kang Kai
2016-02-23  7:25       ` Richard Purdie
2016-02-23  7:31         ` Kang Kai
2016-02-23 16:29   ` Burton, Ross
2016-02-23 17:01     ` Burton, Ross
2016-02-23 17:10       ` Burton, Ross
2016-02-24  8:51         ` Kang Kai

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox