meta/lib/oe/rootfs.py: dont' remove any packages by default

meta/lib/oe/rootfs.py: dont' remove any packages by default

[Robert Yang]

Currently, the rootfs.py removes base-passwd, shadow, update-rc.d,
update-alternatives and run-postinsts when package-management not
in IMAGE_FEATURES, this causes two problems:

1) This makes we can't install the removed pkgs to rootfs, such as
    IMAGE_INSTALL_append = " shadow", the shadow can't installed (first
    installed, then removed)

2) The base-passwd has been removed, but the /etc/passwd and /etc/group
    are still existed since they are generated by preinst, this would
    confuse the user, and we can't add a postuninst to remove /etc/passwd
    and /etc/group since they are required when runtime.

I think that we should not remove any pkgs by default, we can add some
interfaces/ways to let the user decide whether to remove them or any
other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
or only add run-postinsts to it by default.

-- 
Thanks

Robert

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Paul Eggleton]

Hi Robert,

On Friday 31 July 2015 14:56:15 Robert Yang wrote:
> Currently, the rootfs.py removes base-passwd, shadow, update-rc.d,
> update-alternatives and run-postinsts when package-management not
> in IMAGE_FEATURES, this causes two problems:
> 
> 1) This makes we can't install the removed pkgs to rootfs, such as
>     IMAGE_INSTALL_append = " shadow", the shadow can't installed (first
>     installed, then removed)
>
> 2) The base-passwd has been removed, but the /etc/passwd and /etc/group
>     are still existed since they are generated by preinst, this would
>     confuse the user, and we can't add a postuninst to remove /etc/passwd
>     and /etc/group since they are required when runtime.
>
> I think that we should not remove any pkgs by default, we can add some
> interfaces/ways to let the user decide whether to remove them or any
> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
> or only add run-postinsts to it by default.

There is a reason for these not to be there by default when you don't have 
runtime package management - the assumption is you won't be adding or removing 
any users, thus the binaries that those packages install aren't needed.

Now, it might be argued that if for example you're using some non-package-
management based application installation mechanism that has per-application 
users (Android does this) you will need to add and remove users and therefore 
you do still need those tools, in which case we would probably need a 
mechanism for preventing the removal of those packages. I'm not sure whether 
that would be an IMAGE_FEATURES item (e.g. "user-management"), or perhaps we 
just make this code get the value of a variable specifying the list of 
packages to remove instead of it being hardcoded as it is now. Either way 
though I think the default behaviour does make sense for most people and I 
don't think we ought to be changing that part.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Jack Mitchell]

On 31/07/15 11:46, Paul Eggleton wrote:
> Hi Robert,
>
> On Friday 31 July 2015 14:56:15 Robert Yang wrote:
>> Currently, the rootfs.py removes base-passwd, shadow, update-rc.d,
>> update-alternatives and run-postinsts when package-management not
>> in IMAGE_FEATURES, this causes two problems:
>>
>> 1) This makes we can't install the removed pkgs to rootfs, such as
>>      IMAGE_INSTALL_append = " shadow", the shadow can't installed (first
>>      installed, then removed)
>>
>> 2) The base-passwd has been removed, but the /etc/passwd and /etc/group
>>      are still existed since they are generated by preinst, this would
>>      confuse the user, and we can't add a postuninst to remove /etc/passwd
>>      and /etc/group since they are required when runtime.
>>
>> I think that we should not remove any pkgs by default, we can add some
>> interfaces/ways to let the user decide whether to remove them or any
>> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
>> or only add run-postinsts to it by default.
> There is a reason for these not to be there by default when you don't have
> runtime package management - the assumption is you won't be adding or removing
> any users, thus the binaries that those packages install aren't needed.
>
> Now, it might be argued that if for example you're using some non-package-
> management based application installation mechanism that has per-application
> users (Android does this) you will need to add and remove users and therefore
> you do still need those tools, in which case we would probably need a
> mechanism for preventing the removal of those packages. I'm not sure whether
> that would be an IMAGE_FEATURES item (e.g. "user-management"), or perhaps we
> just make this code get the value of a variable specifying the list of
> packages to remove instead of it being hardcoded as it is now. Either way
> though I think the default behaviour does make sense for most people and I
> don't think we ought to be changing that part.
>
> Cheers,
> Paul

Recently I tried (and failed) to add update-rc.d to my image so I could 
disable/enable init scripts easily and this explains why. I don't know 
what to do about it, but I don't have package management however I would 
like update-rc.d, so I second that something needs to be done to allow 
exceptions or overrides.

Cheers,

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Nicolas Dechesne]

On Fri, Jul 31, 2015 at 12:46 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
>> I think that we should not remove any pkgs by default, we can add some
>> interfaces/ways to let the user decide whether to remove them or any
>> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
>> or only add run-postinsts to it by default.
>
> There is a reason for these not to be there by default when you don't have
> runtime package management - the assumption is you won't be adding or removing
> any users, thus the binaries that those packages install aren't needed.


I am not sure to understand the relationship between runtime package
management, and adding/removing users? Why would no runtime package
management, imply no need to add/remove users?

nico

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Paul Eggleton]

Hi Nicolas,

On Friday 31 July 2015 13:09:29 Nicolas Dechesne wrote:
> On Fri, Jul 31, 2015 at 12:46 PM, Paul Eggleton
> 
> <paul.eggleton@linux.intel.com> wrote:
> >> I think that we should not remove any pkgs by default, we can add some
> >> interfaces/ways to let the user decide whether to remove them or any
> >> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
> >> or only add run-postinsts to it by default.
> > 
> > There is a reason for these not to be there by default when you don't have
> > runtime package management - the assumption is you won't be adding or
> > removing any users, thus the binaries that those packages install aren't
> > needed.
>
> I am not sure to understand the relationship between runtime package
> management, and adding/removing users? Why would no runtime package
> management, imply no need to add/remove users?

I guess the underlying assumption is that if you have no package manager, in a 
typical embedded context (if there is such a thing) that implies you won't be 
changing the system after setup.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Nicolas Dechesne]

On Fri, Jul 31, 2015 at 1:12 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
>> I am not sure to understand the relationship between runtime package
>> management, and adding/removing users? Why would no runtime package
>> management, imply no need to add/remove users?
>
> I guess the underlying assumption is that if you have no package manager, in a
> typical embedded context (if there is such a thing) that implies you won't be
> changing the system after setup.

well, that still sounds like 2 separate 'features'.. i think it might
be a good idea to split them out...

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Robert Yang]

On 07/31/2015 06:46 PM, Paul Eggleton wrote:
> Hi Robert,
>
> On Friday 31 July 2015 14:56:15 Robert Yang wrote:
>> Currently, the rootfs.py removes base-passwd, shadow, update-rc.d,
>> update-alternatives and run-postinsts when package-management not
>> in IMAGE_FEATURES, this causes two problems:
>>
>> 1) This makes we can't install the removed pkgs to rootfs, such as
>>      IMAGE_INSTALL_append = " shadow", the shadow can't installed (first
>>      installed, then removed)
>>
>> 2) The base-passwd has been removed, but the /etc/passwd and /etc/group
>>      are still existed since they are generated by preinst, this would
>>      confuse the user, and we can't add a postuninst to remove /etc/passwd
>>      and /etc/group since they are required when runtime.
>>
>> I think that we should not remove any pkgs by default, we can add some
>> interfaces/ways to let the user decide whether to remove them or any
>> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
>> or only add run-postinsts to it by default.
>
> There is a reason for these not to be there by default when you don't have
> runtime package management - the assumption is you won't be adding or removing
> any users, thus the binaries that those packages install aren't needed.
>
> Now, it might be argued that if for example you're using some non-package-
> management based application installation mechanism that has per-application
> users (Android does this) you will need to add and remove users and therefore
> you do still need those tools, in which case we would probably need a
> mechanism for preventing the removal of those packages. I'm not sure whether
> that would be an IMAGE_FEATURES item (e.g. "user-management"), or perhaps we
> just make this code get the value of a variable specifying the list of
> packages to remove instead of it being hardcoded as it is now. Either way

I prefer use a variable like REMOVE_PACKAGS to remove them, but after more
thinking, it seems that we need something like user-management since the
removing only be enabled when package-management/user-management is not
in IMAGE_FEATURES.

On the other side, use IMAGE_FEATURES like user-management may also
cause confusion to the user, when people does things like:

IMAGE_INSTALL_append += "shadow"

He wants to use the command like "useradd", but sorry, the "user-management"
is not in IMAGE_FEATURES, so he can't get the command "useradd". Usually
linux users know that the package like shadow provides the command "useradd",
but only OE developers/users(or advanced developer/users) know that he also
needs user-management in IMAGE_FEATURES, this seems not a good user experience.

The similar to rpm/package-management, when we install rpm package in the image,
usually, we'd like to use command like "rpm -qa/ql", but it doesn't work when
package-management is not in the IMAGE_FEATURES, we've received a few
questions/complains about this.

I think that OE is a development platform, we'd better avoid such policies
for the end user, we don't remove anything for them in the fundamental code
like rootfs.py by default, and leave interfaces for them to remove when
they want.

// Robert

> though I think the default behaviour does make sense for most people and I
> don't think we ought to be changing that part.
>
> Cheers,
> Paul
>

Re: meta/lib/oe/rootfs.py: dont' remove any packages by default

[Paul Eggleton]

On Saturday 01 August 2015 11:57:58 Robert Yang wrote:
> On 07/31/2015 06:46 PM, Paul Eggleton wrote:
> > Hi Robert,
> > 
> > On Friday 31 July 2015 14:56:15 Robert Yang wrote:
> >> Currently, the rootfs.py removes base-passwd, shadow, update-rc.d,
> >> update-alternatives and run-postinsts when package-management not
> >> in IMAGE_FEATURES, this causes two problems:
> >> 
> >> 1) This makes we can't install the removed pkgs to rootfs, such as
> >> 
> >>      IMAGE_INSTALL_append = " shadow", the shadow can't installed (first
> >>      installed, then removed)
> >> 
> >> 2) The base-passwd has been removed, but the /etc/passwd and /etc/group
> >> 
> >>      are still existed since they are generated by preinst, this would
> >>      confuse the user, and we can't add a postuninst to remove
> >>      /etc/passwd
> >>      and /etc/group since they are required when runtime.
> >> 
> >> I think that we should not remove any pkgs by default, we can add some
> >> interfaces/ways to let the user decide whether to remove them or any
> >> other pkgs, for example, add a REMOVE_PACKAGS variable, leave it as NULL
> >> or only add run-postinsts to it by default.
> > 
> > There is a reason for these not to be there by default when you don't have
> > runtime package management - the assumption is you won't be adding or
> > removing any users, thus the binaries that those packages install aren't
> > needed.
> > 
> > Now, it might be argued that if for example you're using some non-package-
> > management based application installation mechanism that has
> > per-application users (Android does this) you will need to add and remove
> > users and therefore you do still need those tools, in which case we would
> > probably need a mechanism for preventing the removal of those packages.
> > I'm not sure whether that would be an IMAGE_FEATURES item (e.g.
> > "user-management"), or perhaps we just make this code get the value of a
> > variable specifying the list of packages to remove instead of it being
> > hardcoded as it is now. Either way
>
> I prefer use a variable like REMOVE_PACKAGS to remove them, but after more
> thinking, it seems that we need something like user-management since the
> removing only be enabled when package-management/user-management is not
> in IMAGE_FEATURES.
>
> On the other side, use IMAGE_FEATURES like user-management may also
> cause confusion to the user, when people does things like:
> 
> IMAGE_INSTALL_append += "shadow"
> 
> He wants to use the command like "useradd", but sorry, the "user-management"
> is not in IMAGE_FEATURES, so he can't get the command "useradd". Usually
> linux users know that the package like shadow provides the command
> "useradd", but only OE developers/users(or advanced developer/users) know
> that he also needs user-management in IMAGE_FEATURES, this seems not a good
> user experience.
>
> The similar to rpm/package-management, when we install rpm package in the
> image, usually, we'd like to use command like "rpm -qa/ql", but it doesn't
> work when package-management is not in the IMAGE_FEATURES, we've received a
> few questions/complains about this.
> 
> I think that OE is a development platform, we'd better avoid such policies
> for the end user, we don't remove anything for them in the fundamental code
> like rootfs.py by default, and leave interfaces for them to remove when
> they want.

Perhaps we need to deconstruct this a bit further. How do these packages get 
into the image when "not needed" in the first place? It's because of explicit 
dependencies added in useradd.bbclass in order to support postinstall scripts. 
If we could make it so those dependencies were somehow elided when package-
management was not in IMAGE_FEATURES rather than explicitly removing the 
packages themselves, maybe that would be a more flexible way of solving the 
issue? Perhaps a (simplistic) approach to doing that would be with RRECOMMENDS 
and BAD_RECOMMENDATIONS.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre