[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

Don't blindly install the /etc/nsswitch.conf file, it's useless for
toolchains that aren't (e)glibc-based and misleading.
Make the installation conditional on a (e)glibc toolchain.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 {system/skeleton/etc => package/glibc}/nsswitch.conf |  0
 toolchain/toolchain.mk                               | 10 ++++++++++
 2 files changed, 10 insertions(+)
 rename {system/skeleton/etc => package/glibc}/nsswitch.conf (100%)

diff --git a/system/skeleton/etc/nsswitch.conf b/package/glibc/nsswitch.conf
similarity index 100%
rename from system/skeleton/etc/nsswitch.conf
rename to package/glibc/nsswitch.conf
diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk
index 8fe06ff..3f3534a 100644
--- a/toolchain/toolchain.mk
+++ b/toolchain/toolchain.mk
@@ -3,6 +3,16 @@
 # TARGET_FINALIZE_HOOKS, to be applied just after all packages
 # have been built.
 
+# Install default nsswitch.conf file if the skeleton doesn't provide it
+ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y)
+define GLIBC_COPY_NSSWITCH_FILE
+	$(Q)if [ ! -f "$(TARGET_DIR)/etc/nsswitch.conf" ]; then \
+		cp -a package/glibc/nsswitch.conf $(TARGET_DIR)/etc; \
+	fi
+endef
+TARGET_FINALIZE_HOOKS += GLIBC_COPY_NSSWITCH_FILE
+endif
+
 # Install the gconv modules
 ifeq ($(BR2_TOOLCHAIN_GLIBC_GCONV_LIBS_COPY),y)
 GCONV_LIBS = $(call qstrip,$(BR2_TOOLCHAIN_GLIBC_GCONV_LIBS_LIST))
-- 
2.0.4

[Buildroot] [PATCH 2/2] glibc/nsswitch.conf: also lookup on dns for networks

[Gustavo Zacarias]

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 package/glibc/nsswitch.conf | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/package/glibc/nsswitch.conf b/package/glibc/nsswitch.conf
index d81e2f0..5c38491 100644
--- a/package/glibc/nsswitch.conf
+++ b/package/glibc/nsswitch.conf
@@ -1,17 +1,13 @@
 # /etc/nsswitch.conf
-#
-# Example configuration of GNU Name Service Switch functionality.
-# If you have the `glibc-doc-reference' and `info' packages installed, try:
-# `info libc "Name Service Switch"' for information about this file.
 
 passwd:         files
 group:          files
 shadow:         files
+
 hosts:          files dns
-networks:       files
+networks:       files dns
 
 protocols:      files
 services:       files
 ethers:         files
 rpc:            files
-
-- 
2.0.4

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 09:28:05 -0300, Gustavo Zacarias wrote:
> Don't blindly install the /etc/nsswitch.conf file, it's useless for
> toolchains that aren't (e)glibc-based and misleading.
> Make the installation conditional on a (e)glibc toolchain.
> 
> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> ---
>  {system/skeleton/etc => package/glibc}/nsswitch.conf |  0
>  toolchain/toolchain.mk                               | 10 ++++++++++
>  2 files changed, 10 insertions(+)
>  rename {system/skeleton/etc => package/glibc}/nsswitch.conf (100%)
> 
> diff --git a/system/skeleton/etc/nsswitch.conf b/package/glibc/nsswitch.conf
> similarity index 100%
> rename from system/skeleton/etc/nsswitch.conf
> rename to package/glibc/nsswitch.conf
> diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk
> index 8fe06ff..3f3534a 100644
> --- a/toolchain/toolchain.mk
> +++ b/toolchain/toolchain.mk
> @@ -3,6 +3,16 @@
>  # TARGET_FINALIZE_HOOKS, to be applied just after all packages
>  # have been built.
>  
> +# Install default nsswitch.conf file if the skeleton doesn't provide it
> +ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y)
> +define GLIBC_COPY_NSSWITCH_FILE
> +	$(Q)if [ ! -f "$(TARGET_DIR)/etc/nsswitch.conf" ]; then \
> +		cp -a package/glibc/nsswitch.conf $(TARGET_DIR)/etc; \

		$(INSTALL) -D -m 0644

maybe ?

> +	fi
> +endef
> +TARGET_FINALIZE_HOOKS += GLIBC_COPY_NSSWITCH_FILE
> +endif

Also, I believe I'd prefer to see this being done in the glibc package
and in the toolchain-external package. I know it's a bit redundant to
have it twice, but I'd like to not have too many files that get
installed through the finalize hooks.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/20/2014 10:02 AM, Thomas Petazzoni wrote:
>> +TARGET_FINALIZE_HOOKS += GLIBC_COPY_NSSWITCH_FILE
>> +endif
> 
> Also, I believe I'd prefer to see this being done in the glibc package
> and in the toolchain-external package. I know it's a bit redundant to
> have it twice, but I'd like to not have too many files that get
> installed through the finalize hooks.

Why? It doesn't write anything unless it's necessary and it's a pretty
small check.
And it's ok to duplicate code for this but not elsewhere? Isn't that
inconsistent?
Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 10:19:49 -0300, Gustavo Zacarias wrote:

> Why? It doesn't write anything unless it's necessary and it's a pretty
> small check.
> And it's ok to duplicate code for this but not elsewhere? Isn't that
> inconsistent?

Well, it's just that I still have my pet project of graphing the size
installed by each package. And for this, every file not installed by a
package is a bit annoying, as we don't know to which package we should
account the installation of the file.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/20/2014 10:27 AM, Thomas Petazzoni wrote:

> Well, it's just that I still have my pet project of graphing the size
> installed by each package. And for this, every file not installed by a
> package is a bit annoying, as we don't know to which package we should
> account the installation of the file.

You've got bigger fish to care about before this tiny file, like
COPY_GCONV_LIBS in toolchain/toolchain.mk, or:

gustavoz at asgard ~/git/buildroot/package $ find . -name \*.mk -exec grep
--with-filename TARGET_FINALIZE {} \;
./perl/perl.mk:TARGET_FINALIZE_HOOKS += PERL_FINALIZE_TARGET
./luarocks/luarocks.mk:TARGET_FINALIZE_HOOKS += LUAROCKS_FINALIZE_TARGET
./google-breakpad/google-breakpad.mk:TARGET_FINALIZE_HOOKS +=
GOOGLE_BREAKPAD_EXTRACT_SYMBOLS
./python/python.mk:TARGET_FINALIZE_HOOKS += PYTHON_FINALIZE_TARGET
./python3/python3.mk:TARGET_FINALIZE_HOOKS += PYTHON3_FINALIZE_TARGET

Besides it's a file that in essence belongs to skeleton, just
conditional on (e)glibc.
You'll have to deal with the other skeleton files as well IMHO, just
treat is as such.
(negative logic could be used, remove if it's not needed, but i don't
think it'll change anything WRT your pet project).
Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 10:34:09 -0300, Gustavo Zacarias wrote:

> You've got bigger fish to care about before this tiny file, like
> COPY_GCONV_LIBS in toolchain/toolchain.mk, or:

Agreed. COPY_GCONV_LIBS could also be move to glibc +
toolchain-external. But we indeed need a way to share the logic to
avoid duplicating it.

> gustavoz at asgard ~/git/buildroot/package $ find . -name \*.mk -exec grep
> --with-filename TARGET_FINALIZE {} \;
> ./perl/perl.mk:TARGET_FINALIZE_HOOKS += PERL_FINALIZE_TARGET

Only removes stuff.

> ./luarocks/luarocks.mk:TARGET_FINALIZE_HOOKS += LUAROCKS_FINALIZE_TARGET

Only removes stuff.

> ./google-breakpad/google-breakpad.mk:TARGET_FINALIZE_HOOKS +=
> GOOGLE_BREAKPAD_EXTRACT_SYMBOLS

Only adds stuff to $(STAGING_DIR), doesn't touch $(TARGET_DIR)

> ./python/python.mk:TARGET_FINALIZE_HOOKS += PYTHON_FINALIZE_TARGET
> ./python3/python3.mk:TARGET_FINALIZE_HOOKS += PYTHON3_FINALIZE_TARGET

Only removes stuff.

> Besides it's a file that in essence belongs to skeleton, just
> conditional on (e)glibc.
> You'll have to deal with the other skeleton files as well IMHO, just
> treat is as such.
> (negative logic could be used, remove if it's not needed, but i don't
> think it'll change anything WRT your pet project).

The skeleton can be made a package of its own.

See, I'm not too far away :-)

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/20/2014 10:37 AM, Thomas Petazzoni wrote:

>> gustavoz at asgard ~/git/buildroot/package $ find . -name \*.mk -exec grep
>> --with-filename TARGET_FINALIZE {} \;
>> ./perl/perl.mk:TARGET_FINALIZE_HOOKS += PERL_FINALIZE_TARGET
> 
> Only removes stuff.

But it will alter your package size measurments outside of package
scope, you need to account for those.

> The skeleton can be made a package of its own.
> 
> See, I'm not too far away :-)

Yes, it's called baselayout in gentoo, base-files in some other distros
and so on, you probably already know them.
But nobody made it yet.
Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 10:43:16 -0300, Gustavo Zacarias wrote:

> >> gustavoz at asgard ~/git/buildroot/package $ find . -name \*.mk -exec grep
> >> --with-filename TARGET_FINALIZE {} \;
> >> ./perl/perl.mk:TARGET_FINALIZE_HOOKS += PERL_FINALIZE_TARGET
> > 
> > Only removes stuff.
> 
> But it will alter your package size measurments outside of package
> scope, you need to account for those.

Yes, this is already taken care of in my script. There are lots of
other files that get removed (header files, static libraries, etc.).

> > The skeleton can be made a package of its own.
> > 
> > See, I'm not too far away :-)
> 
> Yes, it's called baselayout in gentoo, base-files in some other distros
> and so on, you probably already know them.

Yep.

> But nobody made it yet.

Indeed, but it wouldn't be too hard to do I believe.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/20/2014 10:54 AM, Thomas Petazzoni wrote:

>> Yes, it's called baselayout in gentoo, base-files in some other distros
>> and so on, you probably already know them.
> 
> Yep.
> 
>> But nobody made it yet.
> 
> Indeed, but it wouldn't be too hard to do I believe.

Not at all, already done in private initscript revamp, but not in an
upstreamable/generic way since it's not finished, part of my
configurable sysv initscripts (i think i've mentioned the idea on the ML
and/or IRC).
I dunno if Maxime or someone else is working on anything similar since
i'm somewhat outside the loop, and my motivation is pretty low as i've
already said.
Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 11:12:44 -0300, Gustavo Zacarias wrote:

> Not at all, already done in private initscript revamp, but not in an
> upstreamable/generic way since it's not finished, part of my
> configurable sysv initscripts (i think i've mentioned the idea on the ML
> and/or IRC).
> I dunno if Maxime or someone else is working on anything similar since
> i'm somewhat outside the loop, and my motivation is pretty low as i've
> already said.

Well, Maxime is still pushing his initscripts package to make sure a
root filesystem with systemd is not cluttered with an /etc/init.d/
directory. I don't know if he intends to go further than that.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Maxime Hadjinlian]

Hi Thomas, Gustavo, all,

On Mon, Oct 20, 2014 at 4:28 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Gustavo Zacarias,
>
> On Mon, 20 Oct 2014 11:12:44 -0300, Gustavo Zacarias wrote:
>
>> Not at all, already done in private initscript revamp, but not in an
>> upstreamable/generic way since it's not finished, part of my
>> configurable sysv initscripts (i think i've mentioned the idea on the ML
>> and/or IRC).
>> I dunno if Maxime or someone else is working on anything similar since
>> i'm somewhat outside the loop, and my motivation is pretty low as i've
>> already said.
>
> Well, Maxime is still pushing his initscripts package to make sure a
> root filesystem with systemd is not cluttered with an /etc/init.d/
> directory. I don't know if he intends to go further than that.
I would like to have a much better control over what's installed from
the initscripts (or other) package. I don't have, yet, a clear idea of
what I'd like but Gustavo, if you started working on this, maybe I
could help ?
>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux, Kernel and Android engineering
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/22/2014 11:10 AM, Maxime Hadjinlian wrote:

> I would like to have a much better control over what's installed from
> the initscripts (or other) package. I don't have, yet, a clear idea of
> what I'd like but Gustavo, if you started working on this, maybe I
> could help ?

I have nothing big done (at least upstreamable, i've got custom scripts
done for my projects but they're not pretty).
A big wishlist would be:

1) Making /etc/inittab as minimal as possible:

-----
# Startup
::sysinit:/etc/init.d/rcS

# Put a getty on the serial port
#ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100 # GENERIC_SERIAL

# Shutdown
::ctrlaltdel:/etc/init.d/rcK
null::shutdown:/etc/init.d/rcK
-----

Move all the "Startup the system" actions into /etc/init.d/S00sysinit
Move all the shutdown actions into /etc/init.d/K99shutdown

2) Move all of the non-init specific skeleton files into a package
(basefiles, sysfiles, some other pretty name).

3) Make SysV-initscripts more friendly, configurable (basic options,
startup or not).
Ship a default behaviour in /etc/default/$name or some other directory
and make it overridable via a counterpart in /etc/config/$name (or some
other naming convention).
In easily parseable variables, with some common settings, like:
STARTUP=YES
and such.

4) Make /run a separate tmpfs filesystem instead of -> /tmp, it really
looks ugly to find pidfiles in there and might be racy.

5) Make sysvinit without busybox work really well: this needs the
addition of a couple of packages (ifupdown if we plan on doing that for
interfaces for SysV-style), and start-stop-daemon (i don't recall OTOH
which debian package ships it but it's available).
And document what's required - i don't think it's work to select heaven
& hell automatically since it's pretty subjective.

6) For systemd networking take a look at netctl
(https://wiki.archlinux.org/index.php/netctl) which works for
simple/modest setups with simple configuration files.

I don't recall more right now :)
Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Gustavo Zacarias]

On 10/22/2014 04:44 PM, Gustavo Zacarias wrote:

> 5) Make sysvinit without busybox work really well: this needs the
> addition of a couple of packages (ifupdown if we plan on doing that for
> interfaces for SysV-style), and start-stop-daemon (i don't recall OTOH
> which debian package ships it but it's available).
> And document what's required - i don't think it's work to select heaven
> & hell automatically since it's pretty subjective.

s/work/worth/

> I don't recall more right now :)
> Regards.

7) Make a common template for sysv initscripts and update the current
ones - they're pretty divergent right now.

8) Most embedded systems won't shutdown in the distribution sense, but
still we should do K* initscript symlinks where appropiate.

Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Maxime Hadjinlian]

On Wed, Oct 22, 2014 at 9:49 PM, Gustavo Zacarias
<gustavo@zacarias.com.ar> wrote:
> On 10/22/2014 04:44 PM, Gustavo Zacarias wrote:
>
>> 5) Make sysvinit without busybox work really well: this needs the
>> addition of a couple of packages (ifupdown if we plan on doing that for
>> interfaces for SysV-style), and start-stop-daemon (i don't recall OTOH
>> which debian package ships it but it's available).
>> And document what's required - i don't think it's work to select heaven
>> & hell automatically since it's pretty subjective.
>
> s/work/worth/
>
>> I don't recall more right now :)
>> Regards.
>
> 7) Make a common template for sysv initscripts and update the current
> ones - they're pretty divergent right now.
>
> 8) Most embedded systems won't shutdown in the distribution sense, but
> still we should do K* initscript symlinks where appropiate.
I have pasted that in my todo, I'll work on this and send you a link
to a branch so we can work this one out.
>
> Regards.

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Arnout Vandecappelle]

On 20/10/14 15:02, Thomas Petazzoni wrote:
> Dear Gustavo Zacarias,
> 
> On Mon, 20 Oct 2014 09:28:05 -0300, Gustavo Zacarias wrote:
>> Don't blindly install the /etc/nsswitch.conf file, it's useless for
>> toolchains that aren't (e)glibc-based and misleading.
>> Make the installation conditional on a (e)glibc toolchain.
>>
>> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
>> ---
>>  {system/skeleton/etc => package/glibc}/nsswitch.conf |  0
>>  toolchain/toolchain.mk                               | 10 ++++++++++
>>  2 files changed, 10 insertions(+)
>>  rename {system/skeleton/etc => package/glibc}/nsswitch.conf (100%)
>>
>> diff --git a/system/skeleton/etc/nsswitch.conf b/package/glibc/nsswitch.conf
>> similarity index 100%
>> rename from system/skeleton/etc/nsswitch.conf
>> rename to package/glibc/nsswitch.conf
>> diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk
>> index 8fe06ff..3f3534a 100644
>> --- a/toolchain/toolchain.mk
>> +++ b/toolchain/toolchain.mk
>> @@ -3,6 +3,16 @@
>>  # TARGET_FINALIZE_HOOKS, to be applied just after all packages
>>  # have been built.
>>  
>> +# Install default nsswitch.conf file if the skeleton doesn't provide it
>> +ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y)
>> +define GLIBC_COPY_NSSWITCH_FILE
>> +	$(Q)if [ ! -f "$(TARGET_DIR)/etc/nsswitch.conf" ]; then \
>> +		cp -a package/glibc/nsswitch.conf $(TARGET_DIR)/etc; \
> 
> 		$(INSTALL) -D -m 0644
> 
> maybe ?
> 
>> +	fi
>> +endef
>> +TARGET_FINALIZE_HOOKS += GLIBC_COPY_NSSWITCH_FILE
>> +endif
> 
> Also, I believe I'd prefer to see this being done in the glibc package
> and in the toolchain-external package. I know it's a bit redundant to
> have it twice, but I'd like to not have too many files that get
> installed through the finalize hooks.

 After reading the whole thread, I'm with gustavoz on this one.

- If you do it in glibc + toolchain-external, you have duplication.

- We expect a skeleton package to appear. When it exists, this and a few other
finalize things can go there instead. In that case, you have no duplication anymore.

- The patch to move this from a finalize hook to the skeleton package will be a
lot simpler than a patch that moves it out of glibc + toolchain-external. In
addition, it's easier to forget it when it's spread out like that.

- We have this patch now, we don't have a patch that puts it in glibc +
toolchain-external.

 Therefore,

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>


 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1/2] system/skeleton: make nsswitch install conditional

[Thomas Petazzoni]

Dear Gustavo Zacarias,

On Mon, 20 Oct 2014 09:28:05 -0300, Gustavo Zacarias wrote:
> Don't blindly install the /etc/nsswitch.conf file, it's useless for
> toolchains that aren't (e)glibc-based and misleading.
> Make the installation conditional on a (e)glibc toolchain.
> 
> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> ---
>  {system/skeleton/etc => package/glibc}/nsswitch.conf |  0
>  toolchain/toolchain.mk                               | 10 ++++++++++
>  2 files changed, 10 insertions(+)
>  rename {system/skeleton/etc => package/glibc}/nsswitch.conf (100%)

Both patches applied. I've only changed 'cp' to '$(INSTALL) -D -m 0644'
and used a full destination path.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com