[Buildroot] [PATCHv2 0/2] Installation of getent, solving bug #7142

[Buildroot] [PATCHv2 0/2] Installation of getent, solving bug #7142

[Thomas Petazzoni]

Hello,

Here is a second version of the patches to fix bug #7142. As I
proposed, this new version takes a simplified approach: it creates an
autonomous getent package, which simply installs the toolchain's
getent program when the toolchain is glibc based, and otherwise
installed the wrapper script borrowed from uclibc sources. This new
solution avoids doing complicated stuff in the toolchain support, and
completely isolates all the logic in the getent package.

It is functionally identical to the previously proposed solution,
except for internal uClibc toolchains: in the previous solution, the
getent script from the uClibc sources was used, while with this
solution, it's a fixed version inside the getent package that is being
used. However, considering the very simple nature of the script, it's
unlikely to be changed frequently in uClibc sources, so this
difference shouldn't cause much troubles.

Thanks,

Thomas

Thomas Petazzoni (2):
  getent: new package
  ecryptfs-utils: add runtime dependency on getent

 package/Config.in                |  1 +
 package/ecryptfs-utils/Config.in |  2 ++
 package/getent/Config.in         | 10 +++++++++
 package/getent/getent            | 45 ++++++++++++++++++++++++++++++++++++++++
 package/getent/getent.mk         | 26 +++++++++++++++++++++++
 5 files changed, 84 insertions(+)
 create mode 100644 package/getent/Config.in
 create mode 100644 package/getent/getent
 create mode 100644 package/getent/getent.mk

-- 
2.0.0

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas Petazzoni]

The ecryptfs-utils scripts require the 'getent' program to be
installed to find the home directory of users. However, Buildroot
currently never installs this program, and therefore bug #7142 was
reported, explaining that ecryptfs-utils is not working properly.

In normal Linux systems, the getent program is provided by glibc, and
allows to query not only /etc/passwd, but also other NSS databases
such as LDAP and others.

In the context of Buildroot, this gives us several cases:

 1/ Internal toolchain

    a/ glibc/eglibc. In this case, the getent program is already built
       and installed by Buildroot in the staging directory, so the
       only thing missing is installing it in the target directory.

    b/ uclibc. uClibc provides a simple shell script that emulates the
       behavior of getent. It is located in extra/scripts/getent in
       the uClibc sources, but is currently never installed.

    c/ musl. There seems to be no getent implementation, and musl does
       not support NSS.

 2/ External toolchain

    a/ glibc/eglibc. In several external toolchains that we tested,
       there is a pre-built getent binary available in the sysroot,
       but Buildroot is not installing it to the target.

    b/ uclibc. The getent wrapper script is typically not part of any
       external uClibc toolchain.

    c/ musl. There is no getent implementation.

This patch proposes to solve this problem by introducing a getent
package, which has the following behavior:

 - When the toolchain is glibc based (either internal or external), it
   installs the getent program that was built and installed in the
   staging directory. This covers cases 1/ a/ and 2/ a/ above.

 - When the toolchain is uclibc or musl based, it installs a version
   of uclibc's getent wrapper script that is built into the getent
   package. This script is unlikely to change over time, so having it
   directly built into the package should not cause much issues moving
   forward. This covers all other cases above.

This solution allows to install a NSS-capable getent when glibc/eglibc
is used, and otherwise to rely on uClibc's wrapper script.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/Config.in        |  1 +
 package/getent/Config.in | 10 ++++++++++
 package/getent/getent    | 45 +++++++++++++++++++++++++++++++++++++++++++++
 package/getent/getent.mk | 26 ++++++++++++++++++++++++++
 4 files changed, 82 insertions(+)
 create mode 100644 package/getent/Config.in
 create mode 100644 package/getent/getent
 create mode 100644 package/getent/getent.mk

diff --git a/package/Config.in b/package/Config.in
index 4520ba6..c5c47ff 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1159,6 +1159,7 @@ if BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
 endif
 	source "package/dsp-tools/Config.in"
 	source "package/ftop/Config.in"
+	source "package/getent/Config.in"
 	source "package/htop/Config.in"
 	source "package/iprutils/Config.in"
 	source "package/keyutils/Config.in"
diff --git a/package/getent/Config.in b/package/getent/Config.in
new file mode 100644
index 0000000..a7303cb
--- /dev/null
+++ b/package/getent/Config.in
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_GETENT
+	bool "getent"
+	help
+	  This package installs the 'getent' utility, which allows to
+	  get entries from Name Service Switch libraries. For glibc
+	  toolchains, it's the real getent program from the C library
+	  that gets installed, which is NSS-capable. For uclibc and
+	  musl toolchains, it's a simple wrapper script that emulates
+	  getent's behavior, since there is no NSS support in uclibc
+	  and musl.
diff --git a/package/getent/getent b/package/getent/getent
new file mode 100644
index 0000000..fdda793
--- /dev/null
+++ b/package/getent/getent
@@ -0,0 +1,45 @@
+#!/bin/sh
+# $Header: /var/cvs/uClibc/extra/scripts/getent,v 1.2 2005/02/02 14:18:01 solar Exp $
+#
+# Closely (not perfectly) emulate the behavior of glibc's getent utility
+#
+#passwd|shadow|group|aliases|hosts|networks|ethers|netgroup|protocols|services|rpc
+# only returns the first match (by design)
+# dns based search is not supported (hosts,networks)
+# case-insensitive matches not supported (ethers; others?)
+# may return false-positives (hosts,protocols,rpc,services,ethers)
+#
+# Taken from uClibc 0.9.33.
+
+export PATH="${PATH}:/bin:/usr/bin"
+
+file="/etc/$1"
+case $1 in
+	passwd|group)
+		match="^$2:\|^[^:]*:[^:]*:$2:" ;;
+	shadow)
+		match="^$2:" ;;
+	networks|netgroup)
+		match="^[[:space:]]*$2\>" ;;
+	hosts|protocols|rpc|services|ethers)
+		match="\<$2\>" ;;
+	aliases)
+		match="^[[:space:]]*$2[[:space:]]*:" ;;
+	""|-h|--help)
+		echo "USAGE: $0 database [key]"
+		exit 0 ;;
+	*)
+		echo "$0: Unknown database: $1" 1>&2
+		exit 1 ;;
+esac
+
+if [ ! -f "$file" ] ; then
+	echo "$0: Could not find database file for $1" 1>&2
+	exit 1
+fi
+
+if [ $# -eq 1 ] ; then
+	exec cat "$file"
+else
+	sed "s/#.*//; /$match/q; d" "$file" | grep . || exit 2
+fi
diff --git a/package/getent/getent.mk b/package/getent/getent.mk
new file mode 100644
index 0000000..dd08478
--- /dev/null
+++ b/package/getent/getent.mk
@@ -0,0 +1,26 @@
+################################################################################
+#
+# getent
+#
+################################################################################
+
+# source included in Buildroot
+GETENT_SOURCE =
+
+GETENT_VERSION = buildroot-$(BR2_VERSION)
+GETENT_LICENSE = LGPLv2.1+
+
+# For glibc toolchains, we use the getent program built/installed by
+# the C library. For other toolchains, we use the wrapper script
+# included in this package.
+ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y)
+GETENT_LOCATION = $(STAGING_DIR)/usr/bin/getent
+else
+GETENT_LOCATION = package/getent/getent
+endif
+
+define GETENT_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(GETENT_LOCATION) $(TARGET_DIR)/usr/bin/getent
+endef
+
+$(eval $(generic-package))
-- 
2.0.0

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas De Schampheleire]

Hi Thomas,

On Mon, Aug 18, 2014 at 11:54 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
[..]
> diff --git a/package/getent/getent.mk b/package/getent/getent.mk
> new file mode 100644
> index 0000000..dd08478
> --- /dev/null
> +++ b/package/getent/getent.mk
> @@ -0,0 +1,26 @@
> +################################################################################
> +#
> +# getent
> +#
> +################################################################################
> +
> +# source included in Buildroot
> +GETENT_SOURCE =
> +
> +GETENT_VERSION = buildroot-$(BR2_VERSION)
> +GETENT_LICENSE = LGPLv2.1+

How do you conclude this license?
Isn't this dependent on the case glibc versus uclibc/musl ? Or do each
of these use the same license?
Also, shouldn't we specify GETENT_LICENSE_FILES and make sure the
appropriate license text is present?

Best regards,
Thomas

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Mon, 18 Aug 2014 13:50:57 +0200, Thomas De Schampheleire wrote:

> How do you conclude this license?

uclibc: there is no license specific in the script itself, so by
default it's the same license as the entire project, i.e LGPLv2.1+

glibc: the header comment in nss/getent.c is clear: it's under
LGPLv2.1+.

> Isn't this dependent on the case glibc versus uclibc/musl ? Or do each
> of these use the same license?

See above :)

> Also, shouldn't we specify GETENT_LICENSE_FILES and make sure the
> appropriate license text is present?

Which license text file should be used? In neither of the uclibc/musl
or glibc cases we have access to the license text. I can include a
COPYING file in package/getent/, but that's going to be 100x times
larger than the getent script :)

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas De Schampheleire]

On Mon, Aug 18, 2014 at 4:52 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Thomas De Schampheleire,
>
> On Mon, 18 Aug 2014 13:50:57 +0200, Thomas De Schampheleire wrote:
>
>> How do you conclude this license?
>
> uclibc: there is no license specific in the script itself, so by
> default it's the same license as the entire project, i.e LGPLv2.1+
>
> glibc: the header comment in nss/getent.c is clear: it's under
> LGPLv2.1+.
>
>> Isn't this dependent on the case glibc versus uclibc/musl ? Or do each
>> of these use the same license?
>
> See above :)

OK

>
>> Also, shouldn't we specify GETENT_LICENSE_FILES and make sure the
>> appropriate license text is present?
>
> Which license text file should be used? In neither of the uclibc/musl
> or glibc cases we have access to the license text. I can include a
> COPYING file in package/getent/, but that's going to be 100x times
> larger than the getent script :)

Hmm, not sure what to do here.

If we don't specify anything, the developer wanting to distribute an
image will have to manually add a LGPL2.1 license text.

Is there a big problem that the license text would be so much bigger
than the script itself? If we'd have multiple packages with source
included in buildroot, we could move the licenses to one directory to
avoid duplication.

But I have no strong opinion here...

Best regards,
Thomas

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Mon, 18 Aug 2014 17:07:08 +0200, Thomas De Schampheleire wrote:

> > Which license text file should be used? In neither of the uclibc/musl
> > or glibc cases we have access to the license text. I can include a
> > COPYING file in package/getent/, but that's going to be 100x times
> > larger than the getent script :)
> 
> Hmm, not sure what to do here.
> 
> If we don't specify anything, the developer wanting to distribute an
> image will have to manually add a LGPL2.1 license text.
> 
> Is there a big problem that the license text would be so much bigger
> than the script itself? If we'd have multiple packages with source
> included in buildroot, we could move the licenses to one directory to
> avoid duplication.
> 
> But I have no strong opinion here...

Note that we have a lot of packages that have a value for
<pkg>_LICENSE, and no value defined for <pkg>_LICENSE_FILES.

According to http://autobuild.buildroot.org/stats/, we have 76 packages
without <pkg>_LICENSE, and 152 without <pkg>_LICENSE_FILES. Therefore, I
think that's a more global problem, not limited to just this package.
At least, I don't think it should be a blocking issue, especially
considering the fact that this set of patches is meant to fix a bug
before the release.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv2 1/2] getent: new package

[Thomas De Schampheleire]

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> schreef:
>Dear Thomas De Schampheleire,
>
>On Mon, 18 Aug 2014 17:07:08 +0200, Thomas De Schampheleire wrote:
>
>> > Which license text file should be used? In neither of the uclibc/musl
>> > or glibc cases we have access to the license text. I can include a
>> > COPYING file in package/getent/, but that's going to be 100x times
>> > larger than the getent script :)
>> 
>> Hmm, not sure what to do here.
>> 
>> If we don't specify anything, the developer wanting to distribute an
>> image will have to manually add a LGPL2.1 license text.
>> 
>> Is there a big problem that the license text would be so much bigger
>> than the script itself? If we'd have multiple packages with source
>> included in buildroot, we could move the licenses to one directory to
>> avoid duplication.
>> 
>> But I have no strong opinion here...
>
>Note that we have a lot of packages that have a value for
><pkg>_LICENSE, and no value defined for <pkg>_LICENSE_FILES.
>
>According to http://autobuild.buildroot.org/stats/, we have 76 packages
>without <pkg>_LICENSE, and 152 without <pkg>_LICENSE_FILES. Therefore, I
>think that's a more global problem, not limited to just this package.
>At least, I don't think it should be a blocking issue, especially
>considering the fact that this set of patches is meant to fix a bug
>before the release.

No, agreed, let's proceed with this version of the patch...

Best regards,
Thomas

[Buildroot] [PATCHv2 1/2] getent: new package

[Arnout Vandecappelle]

On 18/08/14 11:54, Thomas Petazzoni wrote:
> The ecryptfs-utils scripts require the 'getent' program to be
> installed to find the home directory of users. However, Buildroot
> currently never installs this program, and therefore bug #7142 was
> reported, explaining that ecryptfs-utils is not working properly.
> 
> In normal Linux systems, the getent program is provided by glibc, and
> allows to query not only /etc/passwd, but also other NSS databases
> such as LDAP and others.
> 
> In the context of Buildroot, this gives us several cases:
> 
>  1/ Internal toolchain
> 
>     a/ glibc/eglibc. In this case, the getent program is already built
>        and installed by Buildroot in the staging directory, so the
>        only thing missing is installing it in the target directory.
> 
>     b/ uclibc. uClibc provides a simple shell script that emulates the
>        behavior of getent. It is located in extra/scripts/getent in
>        the uClibc sources, but is currently never installed.
> 
>     c/ musl. There seems to be no getent implementation, and musl does
>        not support NSS.
> 
>  2/ External toolchain
> 
>     a/ glibc/eglibc. In several external toolchains that we tested,
>        there is a pre-built getent binary available in the sysroot,
>        but Buildroot is not installing it to the target.
> 
>     b/ uclibc. The getent wrapper script is typically not part of any
>        external uClibc toolchain.
> 
>     c/ musl. There is no getent implementation.
> 
> This patch proposes to solve this problem by introducing a getent
> package, which has the following behavior:
> 
>  - When the toolchain is glibc based (either internal or external), it
>    installs the getent program that was built and installed in the
>    staging directory. This covers cases 1/ a/ and 2/ a/ above.
> 
>  - When the toolchain is uclibc or musl based, it installs a version
>    of uclibc's getent wrapper script that is built into the getent
>    package. This script is unlikely to change over time, so having it
>    directly built into the package should not cause much issues moving
>    forward. This covers all other cases above.
> 
> This solution allows to install a NSS-capable getent when glibc/eglibc
> is used, and otherwise to rely on uClibc's wrapper script.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Fixes a (run-time) bug, so please apply.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv2 1/2] getent: new package

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 > On 18/08/14 11:54, Thomas Petazzoni wrote:
 >> The ecryptfs-utils scripts require the 'getent' program to be
 >> installed to find the home directory of users. However, Buildroot
 >> currently never installs this program, and therefore bug #7142 was
 >> reported, explaining that ecryptfs-utils is not working properly.
 >> 
 >> This solution allows to install a NSS-capable getent when glibc/eglibc
 >> is used, and otherwise to rely on uClibc's wrapper script.
 >> 
 >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

 > Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 >  Fixes a (run-time) bug, so please apply.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv2 1/2] getent: new package

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 > On 18/08/14 11:54, Thomas Petazzoni wrote:
 >> The ecryptfs-utils scripts require the 'getent' program to be
 >> installed to find the home directory of users. However, Buildroot
 >> currently never installs this program, and therefore bug #7142 was
 >> reported, explaining that ecryptfs-utils is not working properly.
 >> 
 >> In normal Linux systems, the getent program is provided by glibc, and
 >> allows to query not only /etc/passwd, but also other NSS databases
 >> such as LDAP and others.
 >> 
 >> This solution allows to install a NSS-capable getent when glibc/eglibc
 >> is used, and otherwise to rely on uClibc's wrapper script.
 >> 
 >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

 > Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 >  Fixes a (run-time) bug, so please apply.

We have some autobuild issues, like:

http://autobuild.buildroot.net/results/820/8207527e3e42a226649c5f5fbf62fa8a77b89467/

(codesourcery mips 2013.05)

Should we perhaps fallback to the uClibc emulation version if the
toolchain doesn't provide it?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCHv2 1/2] getent: new package

[Arnout Vandecappelle]

On 13/10/14 11:17, Peter Korsgaard wrote:
> >>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
>
>  > On 18/08/14 11:54, Thomas Petazzoni wrote:
>  >> The ecryptfs-utils scripts require the 'getent' program to be
>  >> installed to find the home directory of users. However, Buildroot
>  >> currently never installs this program, and therefore bug #7142 was
>  >> reported, explaining that ecryptfs-utils is not working properly.
>  >>
>  >> In normal Linux systems, the getent program is provided by glibc, and
>  >> allows to query not only /etc/passwd, but also other NSS databases
>  >> such as LDAP and others.
>  >>
>  >> This solution allows to install a NSS-capable getent when glibc/eglibc
>  >> is used, and otherwise to rely on uClibc's wrapper script.
>  >>
>  >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>
>  > Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>
>  >  Fixes a (run-time) bug, so please apply.
>
> We have some autobuild issues, like:
>
> http://autobuild.buildroot.net/results/820/8207527e3e42a226649c5f5fbf62fa8a77b89467/
>
> (codesourcery mips 2013.05)
>
> Should we perhaps fallback to the uClibc emulation version if the
> toolchain doesn't provide it?
>

 The toolchain does provide it, but it's in a different path
(libc/usr/lib/bin/getent).

 Thomas, did you test this at all with a CodeSourcery toolchain because it looks
like it's never in sysroot/usr/bin...

 I guess the external toolchain infrastructure should copy it like it does for
the libraries.

 Regards,
 Arnout

-- 
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv2 2/2] ecryptfs-utils: add runtime dependency on getent

[Thomas Petazzoni]

Fixes bug #7142.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/ecryptfs-utils/Config.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/ecryptfs-utils/Config.in b/package/ecryptfs-utils/Config.in
index 7999ea5..0d912c7 100644
--- a/package/ecryptfs-utils/Config.in
+++ b/package/ecryptfs-utils/Config.in
@@ -5,6 +5,8 @@ config BR2_PACKAGE_ECRYPTFS_UTILS
 	# runtime dependency only, some scripts are using the
 	# 'gettext' program to get translations
 	select BR2_PACKAGE_GETTEXT
+	# runtime dependency only
+	select BR2_PACKAGE_GETENT
 	depends on BR2_USE_WCHAR # gettext
 	depends on BR2_USE_MMU # keyutils
 	depends on !BR2_avr32 && !BR2_microblaze # keyutils
-- 
2.0.0

[Buildroot] [PATCHv2 2/2] ecryptfs-utils: add runtime dependency on getent

[Arnout Vandecappelle]

On 18/08/14 11:54, Thomas Petazzoni wrote:
> Fixes bug #7142.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCHv2 2/2] ecryptfs-utils: add runtime dependency on getent

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 > On 18/08/14 11:54, Thomas Petazzoni wrote:
 >> Fixes bug #7142.
 >> 
 >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

 > Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Committed, thanks.

-- 
Bye, Peter Korsgaard