[PATCH 0/1] shadow-native: disable logging to syslog

[PATCH 0/1] shadow-native: disable logging to syslog

[Scott Garman]

Hello,

This pull request includes a patch to shadow to disable logging to
syslog, to prevent sysroot user and group additions from writing
entries to the host's syslog.

I have build-tested this with core-image-sato (which builds a few
useradd-based recipes, such as avahi and dbus) for all 5 of our
qemu architectures, while watching my syslog to verify that no
useradd or groupadd entries were written. 

Scott

The following changes since commit 1a82989345fb98becb487d270fd93a5e6dffeb47:

  runqemu-internal: Add console=tty for qemuppc and NFS (2012-04-06 01:12:15 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib sgarman/shadow-syslog-fix-oe
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=sgarman/shadow-syslog-fix-oe

Scott Garman (1):
  shadow-native: disable logging to syslog

 .../shadow/files/disable-syslog.patch              |   34 ++++++++++++++++++++
 .../shadow/shadow-native_4.1.4.3.bb                |    5 ++-
 2 files changed, 37 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-extended/shadow/files/disable-syslog.patch

-- 
1.7.5.4

[PATCH 1/1] shadow-native: disable logging to syslog

[Scott Garman]

Disable use of syslog to prevent sysroot user and group additions
from writing entries to the host's syslog.

This fixes [YOCTO #2012]

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
---
 .../shadow/files/disable-syslog.patch              |   34 ++++++++++++++++++++
 .../shadow/shadow-native_4.1.4.3.bb                |    5 ++-
 2 files changed, 37 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-extended/shadow/files/disable-syslog.patch

diff --git a/meta/recipes-extended/shadow/files/disable-syslog.patch b/meta/recipes-extended/shadow/files/disable-syslog.patch
new file mode 100644
index 0000000..1943fd6
--- /dev/null
+++ b/meta/recipes-extended/shadow/files/disable-syslog.patch
@@ -0,0 +1,34 @@
+Disable use of syslog to prevent sysroot user and group additions from
+writing entries to the host's syslog. This patch should only be used
+with the shadow-native recipe.
+
+Upstream-Status: Inappropriate [disable feature]
+
+Signed-off-by: Scott Garman <scott.a.garman@intel.com>
+
+diff -urN shadow-4.1.4.3.orig//src/groupadd.c shadow-4.1.4.3/src/groupadd.c
+--- shadow-4.1.4.3.orig//src/groupadd.c	2011-02-13 09:58:16.000000000 -0800
++++ shadow-4.1.4.3/src/groupadd.c	2012-04-05 10:05:59.440001758 -0700
+@@ -34,6 +34,9 @@
+ 
+ #ident "$Id: groupadd.c 3015 2009-06-05 22:16:56Z nekral-guest $"
+ 
++/* Disable use of syslog since we're running this command against a sysroot */
++#undef USE_SYSLOG
++
+ #include <ctype.h>
+ #include <fcntl.h>
+ #include <getopt.h>
+diff -urN shadow-4.1.4.3.orig//src/useradd.c shadow-4.1.4.3/src/useradd.c
+--- shadow-4.1.4.3.orig//src/useradd.c	2011-02-13 09:58:16.000000000 -0800
++++ shadow-4.1.4.3/src/useradd.c	2012-04-05 10:06:25.076001315 -0700
+@@ -34,6 +34,9 @@
+ 
+ #ident "$Id: useradd.c 3015 2009-06-05 22:16:56Z nekral-guest $"
+ 
++/* Disable use of syslog since we're running this command against a sysroot */
++#undef USE_SYSLOG
++
+ #include <assert.h>
+ #include <ctype.h>
+ #include <errno.h>
diff --git a/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb b/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb
index 42ecc6a..38bb922 100644
--- a/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb
+++ b/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb
@@ -7,14 +7,15 @@ LICENSE = "BSD | Artistic-1.0"
 LIC_FILES_CHKSUM = "file://COPYING;md5=08c553a87d4e51bbed50b20e0adcaede \
                     file://src/passwd.c;firstline=8;endline=30;md5=2899a045e90511d0e043b85a7db7e2fe"
 
-PR = "r3"
+PR = "r4"
 
 SRC_URI = "http://pkg-shadow.alioth.debian.org/releases/${BPN}-${PV}.tar.bz2 \
            file://shadow.automake-1.11.patch \
            file://shadow-4.1.3-dots-in-usernames.patch \
            file://shadow-4.1.4.2-env-reset-keep-locale.patch \
            file://add_root_cmd_options.patch \
-	   file://useradd.patch"
+           file://disable-syslog.patch \
+           file://useradd.patch"
 
 SRC_URI[md5sum] = "b8608d8294ac88974f27b20f991c0e79"
 SRC_URI[sha256sum] = "633f5bb4ea0c88c55f3642c97f9d25cbef74f82e0b4cf8d54e7ad6f9f9caa778" 
-- 
1.7.5.4

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Saul Wold]

On 04/05/2012 11:53 PM, Scott Garman wrote:
> Hello,
>
> This pull request includes a patch to shadow to disable logging to
> syslog, to prevent sysroot user and group additions from writing
> entries to the host's syslog.
>
> I have build-tested this with core-image-sato (which builds a few
> useradd-based recipes, such as avahi and dbus) for all 5 of our
> qemu architectures, while watching my syslog to verify that no
> useradd or groupadd entries were written.
>

With this patch applied, the following error was seen on the AB:

| Running useradd commands...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| WARNING: useradd command did not succeed. Retrying...
| ERROR: tried running useradd command 10 times without success, giving up

Check the AB here:

http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio


Sau!

> Scott
>
> The following changes since commit 1a82989345fb98becb487d270fd93a5e6dffeb47:
>
>    runqemu-internal: Add console=tty for qemuppc and NFS (2012-04-06 01:12:15 +0100)
>
> are available in the git repository at:
>    git://git.pokylinux.org/poky-contrib sgarman/shadow-syslog-fix-oe
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=sgarman/shadow-syslog-fix-oe
>
> Scott Garman (1):
>    shadow-native: disable logging to syslog
>
>   .../shadow/files/disable-syslog.patch              |   34 ++++++++++++++++++++
>   .../shadow/shadow-native_4.1.4.3.bb                |    5 ++-
>   2 files changed, 37 insertions(+), 2 deletions(-)
>   create mode 100644 meta/recipes-extended/shadow/files/disable-syslog.patch
>

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Scott Garman]

On 04/07/2012 04:59 PM, Saul Wold wrote:
> On 04/05/2012 11:53 PM, Scott Garman wrote:
>> Hello,
>>
>> This pull request includes a patch to shadow to disable logging to
>> syslog, to prevent sysroot user and group additions from writing
>> entries to the host's syslog.
>>
>> I have build-tested this with core-image-sato (which builds a few
>> useradd-based recipes, such as avahi and dbus) for all 5 of our
>> qemu architectures, while watching my syslog to verify that no
>> useradd or groupadd entries were written.
>>
>
> With this patch applied, the following error was seen on the AB:
>
> | Running useradd commands...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | ERROR: tried running useradd command 10 times without success, giving up
>
> Check the AB here:
>
> http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio

Hi Saul,

The syslog disable patch cannot trigger this error, I'm pretty certain 
you have encountered another problem.

The useradd class now uses code which checks that the user account or 
group account was created in the passwd and group files, respectively. 
If the account was not created (which is verified via a grep command), 
the script sleeps for 1 second and tries again, up to 10 times. This is 
intended to avoid lockfile races, as useradd and groupadd lock the 
passwd and group files when creating accounts.

It seems extremely unlikely that the passwd file was locked for a full 
10s worth of attempts to access it. I also see from the logs that the 
base-passwd package was installed before this error was encountered, 
which *should* rule out the possibility that the useradd command was 
failing because /etc/passwd didn't exist yet.

Later useradd commands are also failing in this manner, which makes me 
suspect that something is wrong with the /etc/passwd file in this image. 
The groupadd commands, on the other hand, are succeeding without any 
retries.

So it would be helpful for me to know answers to the following:

Was this a build from scratch or from sstate?

Is this problem reproducible? (I'm starting a build from scratch 
overnight on my end)

What does the etc/passwd file in this image look like?

Thanks,

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Chris Larson]

On Sat, Apr 7, 2012 at 4:59 PM, Saul Wold <sgw@linux.intel.com> wrote:
> On 04/05/2012 11:53 PM, Scott Garman wrote:
>>
>> Hello,
>>
>> This pull request includes a patch to shadow to disable logging to
>> syslog, to prevent sysroot user and group additions from writing
>> entries to the host's syslog.
>>
>> I have build-tested this with core-image-sato (which builds a few
>> useradd-based recipes, such as avahi and dbus) for all 5 of our
>> qemu architectures, while watching my syslog to verify that no
>> useradd or groupadd entries were written.
>>
>
> With this patch applied, the following error was seen on the AB:
>
> | Running useradd commands...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | WARNING: useradd command did not succeed. Retrying...
> | ERROR: tried running useradd command 10 times without success, giving up
>
> Check the AB here:
>
> http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio

We've (Mentor) hit this failure, usually from openssh's do_install,
from our automated builds at random over the past week or two.
-- 
Christopher Larson

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Saul Wold]

On 04/08/2012 10:00 PM, Scott Garman wrote:
> On 04/07/2012 04:59 PM, Saul Wold wrote:
>> On 04/05/2012 11:53 PM, Scott Garman wrote:
>>> Hello,
>>>
>>> This pull request includes a patch to shadow to disable logging to
>>> syslog, to prevent sysroot user and group additions from writing
>>> entries to the host's syslog.
>>>
>>> I have build-tested this with core-image-sato (which builds a few
>>> useradd-based recipes, such as avahi and dbus) for all 5 of our
>>> qemu architectures, while watching my syslog to verify that no
>>> useradd or groupadd entries were written.
>>>
>>
>> With this patch applied, the following error was seen on the AB:
>>
>> | Running useradd commands...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | WARNING: useradd command did not succeed. Retrying...
>> | ERROR: tried running useradd command 10 times without success,
>> giving up
>>
>> Check the AB here:
>>
>> http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio
>>
>
> Hi Saul,
>
> The syslog disable patch cannot trigger this error, I'm pretty certain
> you have encountered another problem.
>
> The useradd class now uses code which checks that the user account or
> group account was created in the passwd and group files, respectively.
> If the account was not created (which is verified via a grep command),
> the script sleeps for 1 second and tries again, up to 10 times. This is
> intended to avoid lockfile races, as useradd and groupadd lock the
> passwd and group files when creating accounts.
>
> It seems extremely unlikely that the passwd file was locked for a full
> 10s worth of attempts to access it. I also see from the logs that the
> base-passwd package was installed before this error was encountered,
> which *should* rule out the possibility that the useradd command was
> failing because /etc/passwd didn't exist yet.
>
> Later useradd commands are also failing in this manner, which makes me
> suspect that something is wrong with the /etc/passwd file in this image.
> The groupadd commands, on the other hand, are succeeding without any
> retries.
>
> So it would be helpful for me to know answers to the following:
>
> Was this a build from scratch or from sstate?
>
This was from sstate.

> Is this problem reproducible? (I'm starting a build from scratch
> overnight on my end)
>
Only saw it on one build over the weekend, but turns out a bug already 
existed with this issue, but it was filed as a PAM build failure (see 
2218) , which maybe I need to re-assign to you.


> What does the etc/passwd file in this image look like?
>
You can get it from the AB yourself, correct?  If not, let me know please.

Sau!

> Thanks,
>
> Scott
>

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Scott Garman]

On 04/09/2012 08:30 AM, Saul Wold wrote:
> On 04/08/2012 10:00 PM, Scott Garman wrote:
>> On 04/07/2012 04:59 PM, Saul Wold wrote:
>>> On 04/05/2012 11:53 PM, Scott Garman wrote:
>>>> Hello,
>>>>
>>>> This pull request includes a patch to shadow to disable logging to
>>>> syslog, to prevent sysroot user and group additions from writing
>>>> entries to the host's syslog.
>>>>
>>>> I have build-tested this with core-image-sato (which builds a few
>>>> useradd-based recipes, such as avahi and dbus) for all 5 of our
>>>> qemu architectures, while watching my syslog to verify that no
>>>> useradd or groupadd entries were written.
>>>>
>>>
>>> With this patch applied, the following error was seen on the AB:
>>>
>>> | Running useradd commands...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | WARNING: useradd command did not succeed. Retrying...
>>> | ERROR: tried running useradd command 10 times without success,
>>> giving up
>>>
>>> Check the AB here:
>>>
>>> http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio
>>>
>>>
>>
>> Hi Saul,
>>
>> The syslog disable patch cannot trigger this error, I'm pretty certain
>> you have encountered another problem.
>>
>> The useradd class now uses code which checks that the user account or
>> group account was created in the passwd and group files, respectively.
>> If the account was not created (which is verified via a grep command),
>> the script sleeps for 1 second and tries again, up to 10 times. This is
>> intended to avoid lockfile races, as useradd and groupadd lock the
>> passwd and group files when creating accounts.
>>
>> It seems extremely unlikely that the passwd file was locked for a full
>> 10s worth of attempts to access it. I also see from the logs that the
>> base-passwd package was installed before this error was encountered,
>> which *should* rule out the possibility that the useradd command was
>> failing because /etc/passwd didn't exist yet.
>>
>> Later useradd commands are also failing in this manner, which makes me
>> suspect that something is wrong with the /etc/passwd file in this image.
>> The groupadd commands, on the other hand, are succeeding without any
>> retries.
>>
>> So it would be helpful for me to know answers to the following:
>>
>> Was this a build from scratch or from sstate?
>>
> This was from sstate.
>
>> Is this problem reproducible? (I'm starting a build from scratch
>> overnight on my end)
>>
> Only saw it on one build over the weekend, but turns out a bug already
> existed with this issue, but it was filed as a PAM build failure (see
> 2218) , which maybe I need to re-assign to you.

Yes, I've re-assigned this bug to myself.

>> What does the etc/passwd file in this image look like?
>>
> You can get it from the AB yourself, correct? If not, let me know please.

This was a nightly build and it no longer appears to be on the server - 
assuming I'm connected to the correct one?

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Saul Wold]

On 04/09/2012 09:23 AM, Scott Garman wrote:
> On 04/09/2012 08:30 AM, Saul Wold wrote:
>> On 04/08/2012 10:00 PM, Scott Garman wrote:
>>> On 04/07/2012 04:59 PM, Saul Wold wrote:
>>>> On 04/05/2012 11:53 PM, Scott Garman wrote:
>>>>> Hello,
>>>>>
>>>>> This pull request includes a patch to shadow to disable logging to
>>>>> syslog, to prevent sysroot user and group additions from writing
>>>>> entries to the host's syslog.
>>>>>
>>>>> I have build-tested this with core-image-sato (which builds a few
>>>>> useradd-based recipes, such as avahi and dbus) for all 5 of our
>>>>> qemu architectures, while watching my syslog to verify that no
>>>>> useradd or groupadd entries were written.
>>>>>
>>>>
>>>> With this patch applied, the following error was seen on the AB:
>>>>
>>>> | Running useradd commands...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | WARNING: useradd command did not succeed. Retrying...
>>>> | ERROR: tried running useradd command 10 times without success,
>>>> giving up
>>>>
>>>> Check the AB here:
>>>>
>>>> http://autobuilder.yoctoproject.org:8010/builders/nightly-arm/builds/369/steps/shell_19/logs/stdio
>>>>
>>>>
>>>>
>>>
>>> Hi Saul,
>>>
>>> The syslog disable patch cannot trigger this error, I'm pretty certain
>>> you have encountered another problem.
>>>
>>> The useradd class now uses code which checks that the user account or
>>> group account was created in the passwd and group files, respectively.
>>> If the account was not created (which is verified via a grep command),
>>> the script sleeps for 1 second and tries again, up to 10 times. This is
>>> intended to avoid lockfile races, as useradd and groupadd lock the
>>> passwd and group files when creating accounts.
>>>
>>> It seems extremely unlikely that the passwd file was locked for a full
>>> 10s worth of attempts to access it. I also see from the logs that the
>>> base-passwd package was installed before this error was encountered,
>>> which *should* rule out the possibility that the useradd command was
>>> failing because /etc/passwd didn't exist yet.
>>>
>>> Later useradd commands are also failing in this manner, which makes me
>>> suspect that something is wrong with the /etc/passwd file in this image.
>>> The groupadd commands, on the other hand, are succeeding without any
>>> retries.
>>>
>>> So it would be helpful for me to know answers to the following:
>>>
>>> Was this a build from scratch or from sstate?
>>>
>> This was from sstate.
>>
>>> Is this problem reproducible? (I'm starting a build from scratch
>>> overnight on my end)
>>>
>> Only saw it on one build over the weekend, but turns out a bug already
>> existed with this issue, but it was filed as a PAM build failure (see
>> 2218) , which maybe I need to re-assign to you.
>
> Yes, I've re-assigned this bug to myself.
>
Ok thanks.

>>> What does the etc/passwd file in this image look like?
>>>
>> You can get it from the AB yourself, correct? If not, let me know please.
>
> This was a nightly build and it no longer appears to be on the server -
> assuming I'm connected to the correct one?
>
ab05, since this was a non-lsb build, the tmp dir you need to look at is 
non-lsbtmp, it should be there, I just checked.


> Scott
>

Re: [PATCH 0/1] shadow-native: disable logging to syslog

[Saul Wold]

On 04/05/2012 11:53 PM, Scott Garman wrote:
> Hello,
>
> This pull request includes a patch to shadow to disable logging to
> syslog, to prevent sysroot user and group additions from writing
> entries to the host's syslog.
>
> I have build-tested this with core-image-sato (which builds a few
> useradd-based recipes, such as avahi and dbus) for all 5 of our
> qemu architectures, while watching my syslog to verify that no
> useradd or groupadd entries were written.
>
> Scott
>
> The following changes since commit 1a82989345fb98becb487d270fd93a5e6dffeb47:
>
>    runqemu-internal: Add console=tty for qemuppc and NFS (2012-04-06 01:12:15 +0100)
>
> are available in the git repository at:
>    git://git.pokylinux.org/poky-contrib sgarman/shadow-syslog-fix-oe
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=sgarman/shadow-syslog-fix-oe
>
> Scott Garman (1):
>    shadow-native: disable logging to syslog
>
>   .../shadow/files/disable-syslog.patch              |   34 ++++++++++++++++++++
>   .../shadow/shadow-native_4.1.4.3.bb                |    5 ++-
>   2 files changed, 37 insertions(+), 2 deletions(-)
>   create mode 100644 meta/recipes-extended/shadow/files/disable-syslog.patch
>

Merged into OE-Core

Thanks
	Sau!