[Buildroot] [PATCH] skeleton-init-systemd: Work around for RO root /var/lib not populating

[Buildroot] [PATCH] skeleton-init-systemd: Work around for RO root /var/lib not populating

[Trent Piepho]

When using a RO root with systemd, it is intended that /var/lib should be
populated at boot time by tmpfiles system mirroring it from
/usr/share/factory/var/lib.

However, this will only happen if /var/lib does not already exist at the
time systemd-tmpfiles runs.  If it does exist, then tmpfiles will
(silently) skip it and do nothing.

It turns out /var/lib will exist, because some part of systemd creates
/var/lib/systemd/catalog on boot before tmpfiles runs.

The fix used here is to also create tmpfiles entries for the contents of
/var/lib/* and /var/lib/systemd/*.  This way, when those directories
already exist, the entire tree is not skipped and instead the
not-yet-existing contents of /var/lib and /var/lib/systemd will be still
be mirrored from the factory dir.

And if /var/lib/systemd, or a prefix of that, stops getting created and
does not exist, it'll still mirror properly.

It does cause some warnings from systemd:
systemd[1]: Starting Create Volatile Files and Directories...
systemd-tmpfiles[148]: [/etc/tmpfiles.d/var-factory.conf:7] Duplicate line for path "/var/lib/systemd", ignoring.
systemd-tmpfiles[148]: [/etc/tmpfiles.d/var-factory.conf:8] Duplicate line for path "/var/lib/systemd/coredump", ignoring.

But they can be ignored.

IMHO, I think a better solution would be for systemd-tmpfiles to gain a
"merge tree" operation that is like "C" but doesn't abort if the
destination exists, but rather merges the source into it.

Signed-off-by: Trent Piepho <tpiepho@impinj.com>
---
 package/skeleton-init-systemd/skeleton-init-systemd.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/package/skeleton-init-systemd/skeleton-init-systemd.mk b/package/skeleton-init-systemd/skeleton-init-systemd.mk
index 93bff9d27b..0ff3c774ba 100644
--- a/package/skeleton-init-systemd/skeleton-init-systemd.mk
+++ b/package/skeleton-init-systemd/skeleton-init-systemd.mk
@@ -42,7 +42,10 @@ endef
 define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 	rm -f $(TARGET_DIR)/var
 	mkdir $(TARGET_DIR)/var
-	for i in $(TARGET_DIR)/usr/share/factory/var/*; do \
+	for i in $(TARGET_DIR)/usr/share/factory/var/* \
+		 $(TARGET_DIR)/usr/share/factory/var/lib/* \
+		 $(TARGET_DIR)/usr/share/factory/var/lib/systemd/*; do \
+		[ -e "$${i}" ] || continue; \
 		j="$${i#$(TARGET_DIR)/usr/share/factory}"; \
 		if [ -L "$${i}" ]; then \
 			printf "L+! %s - - - - %s\n" \
-- 
2.14.3

[Buildroot] [PATCH] skeleton-init-systemd: Work around for RO root /var/lib not populating

[Yann E. MORIN]

Trent, All,

On 2018-02-27 14:16 -0800, Trent Piepho spake thusly:
> When using a RO root with systemd, it is intended that /var/lib should be
> populated at boot time by tmpfiles system mirroring it from
> /usr/share/factory/var/lib.
> 
> However, this will only happen if /var/lib does not already exist at the
> time systemd-tmpfiles runs.  If it does exist, then tmpfiles will
> (silently) skip it and do nothing.
> 
> It turns out /var/lib will exist, because some part of systemd creates
> /var/lib/systemd/catalog on boot before tmpfiles runs.
> 
> The fix used here is to also create tmpfiles entries for the contents of
> /var/lib/* and /var/lib/systemd/*.  This way, when those directories
> already exist, the entire tree is not skipped and instead the
> not-yet-existing contents of /var/lib and /var/lib/systemd will be still
> be mirrored from the factory dir.
> 
> And if /var/lib/systemd, or a prefix of that, stops getting created and
> does not exist, it'll still mirror properly.

I believe that we fixed that in another way with that commit:

    6e5df928539 package/skeleton-systemd: invert factory logic

As thus, I've marked this patch as rejected.

If your use-case is still not fixed, please re-send it.

Thanks!

Regards,
Yann E. MORIN.

> It does cause some warnings from systemd:
> systemd[1]: Starting Create Volatile Files and Directories...
> systemd-tmpfiles[148]: [/etc/tmpfiles.d/var-factory.conf:7] Duplicate line for path "/var/lib/systemd", ignoring.
> systemd-tmpfiles[148]: [/etc/tmpfiles.d/var-factory.conf:8] Duplicate line for path "/var/lib/systemd/coredump", ignoring.
> 
> But they can be ignored.
> 
> IMHO, I think a better solution would be for systemd-tmpfiles to gain a
> "merge tree" operation that is like "C" but doesn't abort if the
> destination exists, but rather merges the source into it.
> 
> Signed-off-by: Trent Piepho <tpiepho@impinj.com>
> ---
>  package/skeleton-init-systemd/skeleton-init-systemd.mk | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/package/skeleton-init-systemd/skeleton-init-systemd.mk b/package/skeleton-init-systemd/skeleton-init-systemd.mk
> index 93bff9d27b..0ff3c774ba 100644
> --- a/package/skeleton-init-systemd/skeleton-init-systemd.mk
> +++ b/package/skeleton-init-systemd/skeleton-init-systemd.mk
> @@ -42,7 +42,10 @@ endef
>  define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
>  	rm -f $(TARGET_DIR)/var
>  	mkdir $(TARGET_DIR)/var
> -	for i in $(TARGET_DIR)/usr/share/factory/var/*; do \
> +	for i in $(TARGET_DIR)/usr/share/factory/var/* \
> +		 $(TARGET_DIR)/usr/share/factory/var/lib/* \
> +		 $(TARGET_DIR)/usr/share/factory/var/lib/systemd/*; do \
> +		[ -e "$${i}" ] || continue; \
>  		j="$${i#$(TARGET_DIR)/usr/share/factory}"; \
>  		if [ -L "$${i}" ]; then \
>  			printf "L+! %s - - - - %s\n" \
> -- 
> 2.14.3
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH] skeleton-init-systemd: Work around for RO root /var/lib not populating

[Trent Piepho]

On Sat, 2018-03-31 at 11:10 +0200, Yann E. MORIN wrote:
> On 2018-02-27 14:16 -0800, Trent Piepho spake thusly:
> > When using a RO root with systemd, it is intended that /var/lib should be
> > populated at boot time by tmpfiles system mirroring it from
> > /usr/share/factory/var/lib.
> > 
> > However, this will only happen if /var/lib does not already exist at the
> > time systemd-tmpfiles runs.  If it does exist, then tmpfiles will
> > (silently) skip it and do nothing.
> > 
> > It turns out /var/lib will exist, because some part of systemd creates
> > /var/lib/systemd/catalog on boot before tmpfiles runs.
> > 
> > The fix used here is to also create tmpfiles entries for the contents of
> > /var/lib/* and /var/lib/systemd/*.  This way, when those directories
> > already exist, the entire tree is not skipped and instead the
> > not-yet-existing contents of /var/lib and /var/lib/systemd will be still
> > be mirrored from the factory dir.
> > 
> > And if /var/lib/systemd, or a prefix of that, stops getting created and
> > does not exist, it'll still mirror properly.
> 
> I believe that we fixed that in another way with that commit:
> 
>     6e5df928539 package/skeleton-systemd: invert factory logic
> 
> As thus, I've marked this patch as rejected.
> 
> If your use-case is still not fixed, please re-send it.

This patch is already in master, do you intend to revert it?

It's still necessary as "invert factory logic" addressed a different
issue.

My patch address the issue that populating /var/lib from the factory
using only "C! /var/lib - - - -" will silently fail if anything in the
target boot process has created /var/lib before the above tmpfile.d
line is acted upon.  Which happens to be the case.

"invert factory logic" addresses the issue that relative symlinks must
be different if they are used while in /var vs /usr/share/factory/var,
yet the same symlinks were use in both locations.  It solves this by
not using the links from the factory location.  I had an earlier patch
to address this issue by making the links work from both locations.

[Buildroot] [PATCH] skeleton-init-systemd: Work around for RO root /var/lib not populating

[Yann E. MORIN]

Trent, All,

On 2018-04-02 18:23 +0000, Trent Piepho spake thusly:
> On Sat, 2018-03-31 at 11:10 +0200, Yann E. MORIN wrote:
> > On 2018-02-27 14:16 -0800, Trent Piepho spake thusly:
> > > When using a RO root with systemd, it is intended that /var/lib should be
> > > populated at boot time by tmpfiles system mirroring it from
> > > /usr/share/factory/var/lib.
> > > 
> > > However, this will only happen if /var/lib does not already exist at the
> > > time systemd-tmpfiles runs.  If it does exist, then tmpfiles will
> > > (silently) skip it and do nothing.
> > > 
> > > It turns out /var/lib will exist, because some part of systemd creates
> > > /var/lib/systemd/catalog on boot before tmpfiles runs.
> > > 
> > > The fix used here is to also create tmpfiles entries for the contents of
> > > /var/lib/* and /var/lib/systemd/*.  This way, when those directories
> > > already exist, the entire tree is not skipped and instead the
> > > not-yet-existing contents of /var/lib and /var/lib/systemd will be still
> > > be mirrored from the factory dir.
> > > 
> > > And if /var/lib/systemd, or a prefix of that, stops getting created and
> > > does not exist, it'll still mirror properly.
> > 
> > I believe that we fixed that in another way with that commit:
> > 
> >     6e5df928539 package/skeleton-systemd: invert factory logic
> > 
> > As thus, I've marked this patch as rejected.
> > 
> > If your use-case is still not fixed, please re-send it.
> 
> This patch is already in master, do you intend to revert it?

Ah, right. It was still dangling in Patchwork, and I had forgotten it got
applied.

No, we're not gonna revert it.

Sorry for the confusion.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'