RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 4334 bytes --]

Greetings all,

I've recently been doing some work at Mentor Graphics on read-only-rootfs,
for our purposes, and have done some things which I think may be of use.
I'm looking for comments/thoughts on the approach and potential use of a
pattern generally, and also wish to discuss the volatiles vs tmp files.d
situation.

I have recently created a `meta-ro-rootfs` layer to hold work in this
direction which is not yet in or-core. As is fairly usual, it's a staging
area between us and upstream, not a place for anything to live permanently.
What follows is a summation of the work done there thus far:

- Created a 'systemd-tmpfiles' recipe for non-systemd builds, and also for
native

  - Patched in --sysroot= support for systemd-tmpfiles, to facilitate
running it up front against the filesystem at do_rootfs time the way
read_only_rootfs_hook does with populate-volatiles

  - Added an 'update-tmpfiles' script, which iterates over all files in
/etc/default/volatiles/, converting them to tmpfiles.d config files,
writing them into /usr/lib/tmpfiles.d/, and removing the volatile config
files

  - Reduced the dependencies, via patches and EXTRA_OECONF, as much as
seemed to be viable

- Split out 'systemd-tmpfiles' from 'systemd' package in the main systemd
recipe, equivalent to the standalone one, for builds with `systemd` in
DISTRO_FEATURES

- Created a 'read-only-rootfs-systemd' bbclass, which appends the bits to
run both update-tmpfiles and systemd-tmpfiles against the filesystem at
do_rootfs time for systemd distros. This will need to be altered to not
remove the volatiles config files for non-systemd *images*. I realize there
are many use cases to consider in the sysvinit vs systemd realm, and I
haven't verified that all are satisfied just yet, as systemd is our current
priority for r/o rootfs.

- Added a COMPLEMENTARY_GLOBS config (in the layer.conf for now, though I
may move it into the bbclass) which adds `pkg-volatile` for each `pkg` to
the rootfs if read-only-rootfs is enabled.

- Implemented a prototype configuration for dbus which uses this to support
read-only-rootfs. In the main package, we install a dbus.conf into
/usr/lib/tmpfiles.d/ which ensures /var/lib/dbus exists for the non-r/o
case, without including the path in the package (as including paths within
volatile paths in a binary package will break things). In the dbus-volatile
package, I install a different dbus.conf to /etc/tmpfiles.d/, which
overrides the one in /usr/lib/tmpfiles.d/ (yes, update-alternatives would
have been an option too, and has a certain amount of merit, as
/etc/tmpfiles.d/ is supposed to be reserved for use by the administrator),
and this config ensures that /var/volatile/lib/dbus is created and that
/var/lib/dbus is a symlink which points to it.

I have confirmed that I can create a systemd image with read-only-rootfs
and read-only-rootfs-systemd inherited, and dbus is able to write to the
volatile /var/lib/dbus/ path.

I think this is a step in the right direction, though clearly more needs to
be worked out, but this pattern of using override volatile/tmpfiles
configurations with an additional complementary package should let us
cleanly share binary packages between r/o and non-r/o images.

Now, regarding tmpfiles.d vs volatiles. This is a rather unpleasant
situation currently, as for a recipe to cleanly support both sysvinit and
systemd images, it needs not only both sysv init script and systemd
services (though admittedly the latter is optional with sysv compat), it
also needs to provide both a volatiles config and a tmpfiles.d config if it
needs volatile paths.

The standalone systemd-tmpfiles recipe I've created has a fairly small set
of dependencies: intltool-native, dbus, libcap. Given this, I'd like to
propose transitioning away from populate-volatiles to systemd-tmpfiles,
even for non-systemd images. I think that the update-tmpfiles shell script
may be of use in a transition, but before we even begin discussing a
transition path, I'd like to request comments on the notion of moving to it
at all. Thoughts? Objections?

Thanks,
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 4970 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Colin Walters]

On Wed, 2013-07-24 at 11:54 -0700, Chris Larson wrote:

>   - Patched in --sysroot= support for systemd-tmpfiles, to facilitate
> running it up front against the filesystem at do_rootfs time the way
> read_only_rootfs_hook does with populate-volatiles

How are you handling /run?  Is it still a tmpfs in your model?  If so
are you excluding tmpfiles.d snippets which reference /run at rootfs
time?

It strikes me that a less invasive way to achieve readonly rootfs is
to symlink /var -> /run/var.  Or are you doing that already?

Then you'd probably want to just run through any tmpfiles which
reference /etc and /usr.

> 
> - Implemented a prototype configuration for dbus which uses this to
> support read-only-rootfs. 

Err...for what?  /var/run/dbus/system_bus_socket?  You should just
have /var/run -> /run, and that's solved.

If it's for /var/lib/dbus/machine-id, likewise that should on modern
systems just be a symlink to /etc/machine-id.

Though you do need to figure out whether you want a statically
configured machine ID, or to have one generated dynamically at boot.
These both have tradeoffs; basically, whatever you're doing for the
random seed you should probably also do for the machine ID.

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 3209 bytes --]

On Wed, Jul 24, 2013 at 2:47 PM, Colin Walters <walters@verbum.org> wrote:

> On Wed, 2013-07-24 at 11:54 -0700, Chris Larson wrote:
>
> >   - Patched in --sysroot= support for systemd-tmpfiles, to facilitate
> > running it up front against the filesystem at do_rootfs time the way
> > read_only_rootfs_hook does with populate-volatiles
>
> How are you handling /run?  Is it still a tmpfs in your model?  If so
> are you excluding tmpfiles.d snippets which reference /run at rootfs
> time?
>

Still a tmpfs, yes. I haven't excluded them at this time, which means right
now it ends up with the tmpfs paths populated with content that won't be
seen at runtime, due to being under the tmpfs mount. I'm not certain
whether that's a problem or not, but it's absolutely something to think
about.

It strikes me that a less invasive way to achieve readonly rootfs is
> to symlink /var -> /run/var.  Or are you doing that already?
>
> Then you'd probably want to just run through any tmpfiles which
> reference /etc and /usr.
>

That's not a bad idea, and I haven't tried that yet myself. One concern
with this is, with the current setup, retaining existing regenerated bits
in the filesystem would be just a matter of tweaking the tmpfiles
configuration, whereas doing so if you link the entirety could be more
troublesome. One example of a file that lives in /var but isn't volatile is
the asound.state file shipped by alsa-state for some machines. Another is
the package management database, which while r/o if the rootfs is r/o, can
still have value.

>
> > - Implemented a prototype configuration for dbus which uses this to
> > support read-only-rootfs.
>
> Err...for what?  /var/run/dbus/system_bus_socket?  You should just
> have /var/run -> /run, and that's solved.
>

or-core already has /var/run -> /run, so indeed, that's not an issue.


> If it's for /var/lib/dbus/machine-id, likewise that should on modern
> systems just be a symlink to /etc/machine-id.
>
> Though you do need to figure out whether you want a statically
> configured machine ID, or to have one generated dynamically at boot.
> These both have tradeoffs; basically, whatever you're doing for the
> random seed you should probably also do for the machine ID.


It was indeed for the machine-id. I didn't really look closely at exactly
what was being dealt with, as I just wanted an example to show that the
approach with the additional -volatile package with the override config was
going to pan out. :)

Personally, I think that the best approach is for such things to generate
on boot every time, and treat shipping pre-generated content (possibly just
to improve boot time) as a separate task which a distro could take on. That
way everything works, and isn't exposed to any downsides of using
pregenerated content which is the same for all devices by default, unless
opted in. Another example of this would be the ssh host keys. I'm certainly
open to discussing the tradeoffs of the alternatives, though.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 4247 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Otavio Salvador]

Hello Chris,

It is always good when you come with a 'RFC' thread ;-)

On Wed, Jul 24, 2013 at 3:54 PM, Chris Larson <clarson@kergoth.com> wrote:
...
> The standalone systemd-tmpfiles recipe I've created has a fairly small set
> of dependencies: intltool-native, dbus, libcap. Given this, I'd like to
> propose transitioning away from populate-volatiles to systemd-tmpfiles, even
> for non-systemd images. I think that the update-tmpfiles shell script may be
> of use in a transition, but before we even begin discussing a transition
> path, I'd like to request comments on the notion of moving to it at all.

I think it shouldn't be hard to make an C app or script which could
parse systemd-tmpfiles and use this for non-systemd systems. What do
you think?

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Wed, Jul 24, 2013 at 2:51 PM, Otavio Salvador <otavio@ossystems.com.br>wrote:

> On Wed, Jul 24, 2013 at 3:54 PM, Chris Larson <clarson@kergoth.com> wrote:
> ...
> > The standalone systemd-tmpfiles recipe I've created has a fairly small
> set
> > of dependencies: intltool-native, dbus, libcap. Given this, I'd like to
> > propose transitioning away from populate-volatiles to systemd-tmpfiles,
> even
> > for non-systemd images. I think that the update-tmpfiles shell script
> may be
> > of use in a transition, but before we even begin discussing a transition
> > path, I'd like to request comments on the notion of moving to it at all.
>
> I think it shouldn't be hard to make an C app or script which could
> parse systemd-tmpfiles and use this for non-systemd systems. What do
> you think?
>

I considered this too, but systemd-tmpfiles supports quite a lot more than
volatiles, for example restoring selinux labels. Maybe that sounds fun to
you, but it sounded like more trouble than it was worth to me, given
systemd-tmpfiles can be used without systemd :) But yes, it's definitely
another option to consider.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1782 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Otavio Salvador]

On Wed, Jul 24, 2013 at 7:34 PM, Chris Larson <clarson@kergoth.com> wrote:
>
> On Wed, Jul 24, 2013 at 2:51 PM, Otavio Salvador <otavio@ossystems.com.br>
> wrote:
>>
>> On Wed, Jul 24, 2013 at 3:54 PM, Chris Larson <clarson@kergoth.com> wrote:
>> ...
>> > The standalone systemd-tmpfiles recipe I've created has a fairly small
>> > set
>> > of dependencies: intltool-native, dbus, libcap. Given this, I'd like to
>> > propose transitioning away from populate-volatiles to systemd-tmpfiles,
>> > even
>> > for non-systemd images. I think that the update-tmpfiles shell script
>> > may be
>> > of use in a transition, but before we even begin discussing a transition
>> > path, I'd like to request comments on the notion of moving to it at all.
>>
>> I think it shouldn't be hard to make an C app or script which could
>> parse systemd-tmpfiles and use this for non-systemd systems. What do
>> you think?
>
>
> I considered this too, but systemd-tmpfiles supports quite a lot more than
> volatiles, for example restoring selinux labels. Maybe that sounds fun to
> you, but it sounded like more trouble than it was worth to me, given
> systemd-tmpfiles can be used without systemd :) But yes, it's definitely
> another option to consider.


--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Otavio Salvador]

On Wed, Jul 24, 2013 at 7:34 PM, Chris Larson <clarson@kergoth.com> wrote:
>
> On Wed, Jul 24, 2013 at 2:51 PM, Otavio Salvador <otavio@ossystems.com.br>
> wrote:
>>
>> On Wed, Jul 24, 2013 at 3:54 PM, Chris Larson <clarson@kergoth.com> wrote:
>> ...
>> > The standalone systemd-tmpfiles recipe I've created has a fairly small
>> > set
>> > of dependencies: intltool-native, dbus, libcap. Given this, I'd like to
>> > propose transitioning away from populate-volatiles to systemd-tmpfiles,
>> > even
>> > for non-systemd images. I think that the update-tmpfiles shell script
>> > may be
>> > of use in a transition, but before we even begin discussing a transition
>> > path, I'd like to request comments on the notion of moving to it at all.
>>
>> I think it shouldn't be hard to make an C app or script which could
>> parse systemd-tmpfiles and use this for non-systemd systems. What do
>> you think?
>
>
> I considered this too, but systemd-tmpfiles supports quite a lot more than
> volatiles, for example restoring selinux labels. Maybe that sounds fun to
> you, but it sounded like more trouble than it was worth to me, given
> systemd-tmpfiles can be used without systemd :) But yes, it's definitely
> another option to consider.

Sorry, pressed the send button too fast ...

I understand but I think we could solve the tmp files issue only;
people using sysv are expected to solve all those other issues in
another way so it is not a regression for them.

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 2508 bytes --]

On Wed, Jul 24, 2013 at 3:40 PM, Otavio Salvador <otavio@ossystems.com.br>wrote:

> On Wed, Jul 24, 2013 at 7:34 PM, Chris Larson <clarson@kergoth.com> wrote:
> >
> > On Wed, Jul 24, 2013 at 2:51 PM, Otavio Salvador <
> otavio@ossystems.com.br>
> > wrote:
> >>
> >> On Wed, Jul 24, 2013 at 3:54 PM, Chris Larson <clarson@kergoth.com>
> wrote:
> >> ...
> >> > The standalone systemd-tmpfiles recipe I've created has a fairly small
> >> > set
> >> > of dependencies: intltool-native, dbus, libcap. Given this, I'd like
> to
> >> > propose transitioning away from populate-volatiles to
> systemd-tmpfiles,
> >> > even
> >> > for non-systemd images. I think that the update-tmpfiles shell script
> >> > may be
> >> > of use in a transition, but before we even begin discussing a
> transition
> >> > path, I'd like to request comments on the notion of moving to it at
> all.
> >>
> >> I think it shouldn't be hard to make an C app or script which could
> >> parse systemd-tmpfiles and use this for non-systemd systems. What do
> >> you think?
> >
> >
> > I considered this too, but systemd-tmpfiles supports quite a lot more
> than
> > volatiles, for example restoring selinux labels. Maybe that sounds fun to
> > you, but it sounded like more trouble than it was worth to me, given
> > systemd-tmpfiles can be used without systemd :) But yes, it's definitely
> > another option to consider.
>
> Sorry, pressed the send button too fast ...
>
> I understand but I think we could solve the tmp files issue only;
> people using sysv are expected to solve all those other issues in
> another way so it is not a regression for them.


I think those other issues still apply to tmpfiles, just not about their
creation alone :)

Assuming others have other ways of handling that, it does bother me a
little still duplicating that information between tmpfiles config and
whatever that 'another way' was, but it would definitely be a step in the
right direction.

Using a single config file but different implementations would absolutely
be an improvement over the current handling, and would be less invasive,
and wouldn't add the potential additional deps brought in by
systemd-tmpfiles. It's a good idea, and I'd be curious to hear what others
think about this option vs using systemd-tmpfiles itself.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3315 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 5537 bytes --]

Hi Chris,

I'm now working on some bugs related to read-only rootfs.

You can get more information from the bug link below. The related bugs 
are listed in the blocks list of this bug.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=4103
You can also review the patchset for these bugs on

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/read-only-rootfs-in-live-images


I'm really interested on this topic. From my understanding, this RFC 
mainly focuses on read-only rootfs support for systemd based images, right?
I've read though this thread carefully, but still can't get a clear 
picture about this RFC.
Could you please give me a link to your patchset, if convenient?

Best Regards,
Chen Qi

On 07/25/2013 02:54 AM, Chris Larson wrote:
> Greetings all,
>
> I've recently been doing some work at Mentor Graphics on 
> read-only-rootfs, for our purposes, and have done some things which I 
> think may be of use. I'm looking for comments/thoughts on the approach 
> and potential use of a pattern generally, and also wish to discuss the 
> volatiles vs tmp files.d situation.
>
> I have recently created a `meta-ro-rootfs` layer to hold work in this 
> direction which is not yet in or-core. As is fairly usual, it's a 
> staging area between us and upstream, not a place for anything to live 
> permanently. What follows is a summation of the work done there thus far:
>
> - Created a 'systemd-tmpfiles' recipe for non-systemd builds, and also 
> for native
>
>   - Patched in --sysroot= support for systemd-tmpfiles, to facilitate 
> running it up front against the filesystem at do_rootfs time the way 
> read_only_rootfs_hook does with populate-volatiles
>
>   - Added an 'update-tmpfiles' script, which iterates over all files 
> in /etc/default/volatiles/, converting them to tmpfiles.d config 
> files, writing them into /usr/lib/tmpfiles.d/, and removing the 
> volatile config files
>
>   - Reduced the dependencies, via patches and EXTRA_OECONF, as much as 
> seemed to be viable
>
> - Split out 'systemd-tmpfiles' from 'systemd' package in the main 
> systemd recipe, equivalent to the standalone one, for builds with 
> `systemd` in DISTRO_FEATURES
>
> - Created a 'read-only-rootfs-systemd' bbclass, which appends the bits 
> to run both update-tmpfiles and systemd-tmpfiles against the 
> filesystem at do_rootfs time for systemd distros. This will need to be 
> altered to not remove the volatiles config files for non-systemd 
> *images*. I realize there are many use cases to consider in the 
> sysvinit vs systemd realm, and I haven't verified that all are 
> satisfied just yet, as systemd is our current priority for r/o rootfs.
>
> - Added a COMPLEMENTARY_GLOBS config (in the layer.conf for now, 
> though I may move it into the bbclass) which adds `pkg-volatile` for 
> each `pkg` to the rootfs if read-only-rootfs is enabled.
>
> - Implemented a prototype configuration for dbus which uses this to 
> support read-only-rootfs. In the main package, we install a dbus.conf 
> into /usr/lib/tmpfiles.d/ which ensures /var/lib/dbus exists for the 
> non-r/o case, without including the path in the package (as including 
> paths within volatile paths in a binary package will break things). In 
> the dbus-volatile package, I install a different dbus.conf to 
> /etc/tmpfiles.d/, which overrides the one in /usr/lib/tmpfiles.d/ 
> (yes, update-alternatives would have been an option too, and has a 
> certain amount of merit, as /etc/tmpfiles.d/ is supposed to be 
> reserved for use by the administrator), and this config ensures that 
> /var/volatile/lib/dbus is created and that /var/lib/dbus is a symlink 
> which points to it.
>
> I have confirmed that I can create a systemd image with 
> read-only-rootfs and read-only-rootfs-systemd inherited, and dbus is 
> able to write to the volatile /var/lib/dbus/ path.
>
> I think this is a step in the right direction, though clearly more 
> needs to be worked out, but this pattern of using override 
> volatile/tmpfiles configurations with an additional complementary 
> package should let us cleanly share binary packages between r/o and 
> non-r/o images.
>
> Now, regarding tmpfiles.d vs volatiles. This is a rather unpleasant 
> situation currently, as for a recipe to cleanly support both sysvinit 
> and systemd images, it needs not only both sysv init script and 
> systemd services (though admittedly the latter is optional with sysv 
> compat), it also needs to provide both a volatiles config and a 
> tmpfiles.d config if it needs volatile paths.
>
> The standalone systemd-tmpfiles recipe I've created has a fairly small 
> set of dependencies: intltool-native, dbus, libcap. Given this, I'd 
> like to propose transitioning away from populate-volatiles to 
> systemd-tmpfiles, even for non-systemd images. I think that the 
> update-tmpfiles shell script may be of use in a transition, but before 
> we even begin discussing a transition path, I'd like to request 
> comments on the notion of moving to it at all. Thoughts? Objections?
>
> Thanks,
> -- 
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


[-- Attachment #2: Type: text/html, Size: 8554 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 3714 bytes --]

On Wed, Jul 24, 2013 at 7:51 PM, ChenQi <Qi.Chen@windriver.com> wrote:

> You can get more information from the bug link below. The related bugs are
> listed in the blocks list of this bug.
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=4103
> You can also review the patchset for these bugs on
>
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/read-only-rootfs-in-live-images
>
>
> I'm really interested on this topic. From my understanding, this RFC
> mainly focuses on read-only rootfs support for systemd based images, right?
>

It does, but only because of the reliance upon tmpfiles.d's ability to
override config files (the foo-volatile .conf will override the non-r/o foo
package tmpfiles.d config). An alternative approach would be to use
update-alternatives to manage the volatile configuration, and then it could
use either tmpfiles.d or volatiles, but the approach (Providing a r/o
volatile config in a foo-volatile package, and using COMPLEMENTARY_GLOBS to
get them installed for read-only-rootfs) should work, I think, regardless.


> I've read though this thread carefully, but still can't get a clear
> picture about this RFC.
> Could you please give me a link to your patchset, if convenient?
>

To summarize the RFC:

- I think the best approach to use the same set of packages for both
non-r/o and for r/o is to use a complementary package which overrides the
main package configuration to set up the additional links necessary to move
bits into volatile paths. I think that implementing it in this way has
value as opposed to the implementation in "initscripts: use a uniform way
to handle directories in read-only rootfs", as it uses the same mechanism
for read-only as is used for non-read-only, the only difference is the
configuration used and the fact that it gets run at do_rootfs time rather
than at boot, and it means no read-only bits are in the filesystem unless
read-only-rootfs is in IMAGE_FEATURES.
- To support systemd images with read-only-rootfs, we need to be able to
populate tmpfiles.d based on its config files at do_rootfs time the way we
do with populate-volatiles, or we need to duplicate all tmpfiles.d configs
to volatile configs. I chose the former, by enabling a standalone build of
systemd-tmpfiles-native and adding it to the read_only_rootfs_hook.
- https://github.com/MentorEmbedded/meta-ro-rootfs has our work so far on
the above.
- Proposing the need to consolidate tmpfiles.d and volatiles.

- Next steps, from my perspective:

    - We need to agree on a pattern to follow for the read only rootfs
support for individual pieces of recipes, as we need to work through them
individually. Currently we have two possibilities on the floor, the
/etc/default/readonly/ approach advocated by Chen Qi, and the overriding
the volatiles/tmpfiles.d config file approach advocated by myself.
    - We need a plan of attack to avoid the need to duplicate tmpfiles.d
and volatiles configuration. I was leaning toward using systemd-tmpfiles
for both sysvinit and systemd, but others have advocated moving to
tmpfiles.d configuration but using a homerolled script to implement it.
Either would be a step in the right direction: moving away from volatiles
config files in favor of tmpfiles.d config files.
    - We need a coherent, consistent mechanism for the user to choose
whether a given file or files will be pregenerated at build time (e.g. ssh
host keys, machine id) or generated every time at boot, for applicable
cases.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 5068 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 863 bytes --]

On Tue, Jul 30, 2013 at 11:17 AM, Chris Larson <clarson@kergoth.com> wrote:

> I've read though this thread carefully, but still can't get a clear
> picture about this RFC.
> Could you please give me a link to your patchset, if convenient?
>

This is interesting, I didn't notice this in your branch until just now:

"Make this directory writable in case of a read-only rootfs.
Note that if the ssh keys are pregenerated, they will not be lost,
as there's a copying process before bind mounting."

That's quite interesting, and is a definite point in the favor of this
approach, as that's something that isn't really easily doable via
tmpfiles.d and symlinks, only via the bind mounts.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1476 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1909 bytes --]

On Tue, Jul 30, 2013 at 11:17 AM, Chris Larson <clarson@kergoth.com> wrote:

> - Next steps, from my perspective:
>
>     - We need to agree on a pattern to follow for the read only rootfs
> support for individual pieces of recipes, as we need to work through them
> individually. Currently we have two possibilities on the floor, the
> /etc/default/readonly/ approach advocated by Chen Qi, and the overriding
> the volatiles/tmpfiles.d config file approach advocated by myself.
>     - We need a plan of attack to avoid the need to duplicate tmpfiles.d
> and volatiles configuration. I was leaning toward using systemd-tmpfiles
> for both sysvinit and systemd, but others have advocated moving to
> tmpfiles.d configuration but using a homerolled script to implement it.
> Either would be a step in the right direction: moving away from volatiles
> config files in favor of tmpfiles.d config files.
>     - We need a coherent, consistent mechanism for the user to choose
> whether a given file or files will be pregenerated at build time (e.g. ssh
> host keys, machine id) or generated every time at boot, for applicable
> cases.
>

I forgot to mention, we also need to focus on improving the oe-core systemd
support, and the feature parity between sysvinit and systemd images. Some
are missing systemd support at all still (e.g. dropbear's service files are
still in meta-systemd), and others don't behave the same between the two.
For example, were we to pull over those dropbear service files, the
dropbear sysvinit script would behave differently than the service files,
and dropbear would behave differently than openssh.

I don't know if that's a priority for the Yocto 1.5 release, but it
probably should be.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2420 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Paul Eggleton]

On Tuesday 30 July 2013 11:26:58 Chris Larson wrote:
> I forgot to mention, we also need to focus on improving the oe-core systemd
> support, and the feature parity between sysvinit and systemd images. Some
> are missing systemd support at all still (e.g. dropbear's service files are
> still in meta-systemd), and others don't behave the same between the two.
> For example, were we to pull over those dropbear service files, the
> dropbear sysvinit script would behave differently than the service files,
> and dropbear would behave differently than openssh.
> 
> I don't know if that's a priority for the Yocto 1.5 release, but it
> probably should be.

Thanks for bringing this up. There's a bug open for this, but I'd like to 
stress that people need not wait for someone else to get around to it. There 
have already been some patches from Mentor folks to start getting this support 
moved across (although not yet merged due to earlier lack of a concrete 
resolution on the issue of unpackaged unit files, which is now resolved). 
Further help from anyone would be very much appreciated:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=4420

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 5953 bytes --]

On 07/31/2013 02:17 AM, Chris Larson wrote:
> On Wed, Jul 24, 2013 at 7:51 PM, ChenQi <Qi.Chen@windriver.com 
> <mailto:Qi.Chen@windriver.com>> wrote:
>
>     You can get more information from the bug link below. The related
>     bugs are listed in the blocks list of this bug.
>     https://bugzilla.yoctoproject.org/show_bug.cgi?id=4103
>     You can also review the patchset for these bugs on
>
>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/read-only-rootfs-in-live-images
>
>
>     I'm really interested on this topic. From my understanding, this
>     RFC mainly focuses on read-only rootfs support for systemd based
>     images, right?
>
>
> It does, but only because of the reliance upon tmpfiles.d's ability to 
> override config files (the foo-volatile .conf will override the 
> non-r/o foo package tmpfiles.d config). An alternative approach would 
> be to use update-alternatives to manage the volatile configuration, 
> and then it could use either tmpfiles.d or volatiles, but the approach 
> (Providing a r/o volatile config in a foo-volatile package, and using 
> COMPLEMENTARY_GLOBS to get them installed for read-only-rootfs) should 
> work, I think, regardless.
>
>     I've read though this thread carefully, but still can't get a
>     clear picture about this RFC.
>     Could you please give me a link to your patchset, if convenient?
>
>
> To summarize the RFC:
>
> - I think the best approach to use the same set of packages for both 
> non-r/o and for r/o is to use a complementary package which overrides 
> the main package configuration to set up the additional links 
> necessary to move bits into volatile paths. I think that implementing 
> it in this way has value as opposed to the implementation in 
> "initscripts: use a uniform way to handle directories in read-only 
> rootfs", as it uses the same mechanism for read-only as is used for 
> non-read-only, the only difference is the configuration used and the 
> fact that it gets run at do_rootfs time rather than at boot, and it 
> means no read-only bits are in the filesystem unless read-only-rootfs 
> is in IMAGE_FEATURES.
> - To support systemd images with read-only-rootfs, we need to be able 
> to populate tmpfiles.d based on its config files at do_rootfs time the 
> way we do with populate-volatiles, or we need to duplicate all 
> tmpfiles.d configs to volatile configs. I chose the former, by 
> enabling a standalone build of systemd-tmpfiles-native and adding it 
> to the read_only_rootfs_hook.
> - https://github.com/MentorEmbedded/meta-ro-rootfs has our work so far 
> on the above.
> - Proposing the need to consolidate tmpfiles.d and volatiles.
>
Hi Chirst,

I've read the patches. But I haven't tried them out yet because 
currently I can't afford the time ....
 From my understanding, the core concept is to use COMPLEMENTARY_GLOBS 
for *-volatile packages, and these  -volatile packages basically contain 
config files whose major purpose is to create appropriate links at 
rootfs time,  and these packages will be installed only if 
'read-only-rootfs' is in IMAGE_FEATURES. Right?

> - Next steps, from my perspective:
>
>     - We need to agree on a pattern to follow for the read only rootfs 
> support for individual pieces of recipes, as we need to work through 
> them individually. Currently we have two possibilities on the floor, 
> the /etc/default/readonly/ approach advocated by Chen Qi,

When I first came up with approach, I expected that there should be a 
bunch of packages that might need this. But it turned out that in 
OE-core, only the initscripts package needed this. So let's call it a 
fall-back approach. If some package *really* needs this, then we write a 
config file under /etc/default/readonly. By really, I mean, maybe some 
packages could simply be patched to work in a read-only rootfs. And note 
that I added a function in read-only-rootfs-hook.sh which is used to 
check whether a specific directory is on a read-only partition. This is 
because it's possible that users may use customized fstab which, for 
example, mounts a writable media over /var. We should take this 
situation into consideration.

That's also partly the reason that I dropped my 'lighttpd: xxx' patch. 
Moving its log location to some volatile storage to omit errors at 
system boot-up doesn't make the situation any better, because the /www 
directory is still not writable. So I would assume that if a user if 
using a read-only rootfs and he's using his system to hold a web, he 
should at least have some writable storage to store the data.


> and the overriding the volatiles/tmpfiles.d config file approach 
> advocated by myself.
>     - We need a plan of attack to avoid the need to duplicate 
> tmpfiles.d and volatiles configuration. I was leaning toward using 
> systemd-tmpfiles for both sysvinit and systemd,

I have very limited knowledge on systemd and sysvinit, but still I'm not 
sure about this change.
In my opinion, maybe we should make sysv and systemd have as little 
effect on each other as possible.
I'm afraid that this change will bring some potential problems that are 
not spotted at first.

Best Regards,
Chen Qi

> but others have advocated moving to tmpfiles.d configuration but using 
> a homerolled script to implement it. Either would be a step in the 
> right direction: moving away from volatiles config files in favor of 
> tmpfiles.d config files.
>     - We need a coherent, consistent mechanism for the user to choose 
> whether a given file or files will be pregenerated at build time (e.g. 
> ssh host keys, machine id) or generated every time at boot, for 
> applicable cases.
> -- 
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics


[-- Attachment #2: Type: text/html, Size: 9639 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]

On Wed, Jul 24, 2013 at 7:51 PM, ChenQi <Qi.Chen@windriver.com> wrote:

> You can get more information from the bug link below. The related bugs are
> listed in the blocks list of this bug.
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=4103
> You can also review the patchset for these bugs on
>
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/read-only-rootfs-in-live-images
>
>
I'll start playing around with this locally, to see how well it'd work
(with adjustments to support systemd) for our use cases, and will go from
there on our end. Up next for me:

- see if your r/o bits will be suitable for us with systemd rather than
using our tmpfiles override mechanism. we need to support this on much more
than just live images, and we need systemd :)
- pursue systemd support for services missing that support in oe-core
- pursue improved sysvinit / systemd feature parity, and potential
volatiles -> tmpfiles migration / consolidation
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1837 bytes --]

Re: RFC: meta-ro-rootfs approach and volatiles vs tmpfiles.d

[ChenQi]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]

On 07/31/2013 02:53 AM, Chris Larson wrote:
> On Wed, Jul 24, 2013 at 7:51 PM, ChenQi <Qi.Chen@windriver.com 
> <mailto:Qi.Chen@windriver.com>> wrote:
>
>     You can get more information from the bug link below. The related
>     bugs are listed in the blocks list of this bug.
>     https://bugzilla.yoctoproject.org/show_bug.cgi?id=4103
>     You can also review the patchset for these bugs on
>
>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/read-only-rootfs-in-live-images
>
>
> I'll start playing around with this locally, to see how well it'd work 
> (with adjustments to support systemd) for our use cases, and will go 
> from there on our end. Up next for me:
>

Thank you very much for your review :)

Best Regards,
Chen Qi

> - see if your r/o bits will be suitable for us with systemd rather 
> than using our tmpfiles override mechanism. we need to support this on 
> much more than just live images, and we need systemd :)
> - pursue systemd support for services missing that support in oe-core
> - pursue improved sysvinit / systemd feature parity, and potential 
> volatiles -> tmpfiles migration / consolidation
> -- 
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics


[-- Attachment #2: Type: text/html, Size: 3157 bytes --]