* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox
@ 2017-12-13 13:01 Thomas Petazzoni
2017-12-13 13:01 ` [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages Thomas Petazzoni
` (2 more replies)
0 siblings, 3 replies; 26+ messages in thread
From: Thomas Petazzoni @ 2017-12-13 13:01 UTC (permalink / raw)
To: buildroot
Hello,
As we discussed during the Prague Buildroot Developers meeting, in
order to implement per-package SDK, we need to ensure that no package
overwrites files installed by another package.
This RFC series is an attempt at solving this problem for Busybox. I
have not fixed all packages yet: since it is a very boring task to do,
I wanted to first get some feedback on whether the approach looks
reasonable or not.
If the feedback is positive, I'll go ahead and submit proper patches
that fix all packages that conflict with Busybox.
Thanks for your feedback!
Thomas
Thomas Petazzoni (2):
busybox: avoid conflict with other packages
packages: drop no longer needed busybox dependencies
package/bc/bc.mk | 5 ----
package/binutils/binutils.mk | 5 ----
package/busybox/busybox.mk | 58 ++++++++++++++++++++++++++++++++++++++
package/coreutils/coreutils.mk | 6 ----
package/cpio/cpio.mk | 1 -
package/dcron/dcron.mk | 5 ----
package/debianutils/debianutils.mk | 2 --
package/diffutils/diffutils.mk | 4 ---
package/fbset/fbset.mk | 5 ----
package/kmod/kmod.mk | 3 --
package/util-linux/util-linux.mk | 6 ----
11 files changed, 58 insertions(+), 42 deletions(-)
--
2.14.3
^ permalink raw reply [flat|nested] 26+ messages in thread* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-13 13:01 [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Thomas Petazzoni @ 2017-12-13 13:01 ` Thomas Petazzoni 2017-12-13 14:43 ` Baruch Siach 2017-12-13 13:01 ` [Buildroot] [RFC 2/2] packages: drop no longer needed busybox dependencies Thomas Petazzoni 2017-12-28 17:00 ` [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Yann E. MORIN 2 siblings, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-13 13:01 UTC (permalink / raw) To: buildroot One of the requirement to implement per-package SDK is that a package should not overwrite files installed by another package. The busybox package creates such a situation with numerous other packages, because it provides applets that are also provided as full-featured programs in other packages. In order to avoid having other packages overwrite the Busybox applet symbolic link with their own variant, this commit improves the logic in busybox.mk to disable the applets when the corresponding full-featured program will be installed by a different package. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- Note: the list is not complete, this is just a RFC version to find out if that's the direction we want to take. --- package/busybox/busybox.mk | 58 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/package/busybox/busybox.mk b/package/busybox/busybox.mk index 8b720b30b8..e51dbc4887 100644 --- a/package/busybox/busybox.mk +++ b/package/busybox/busybox.mk @@ -250,6 +250,63 @@ endef BUSYBOX_DEPENDENCIES += linux-pam endif +# Avoid collision with other packages +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_BC) += dc +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_BINUTILS) += ar strings +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_CPIO) += cpio +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DCRON) += crond crontab +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DEBIANUTILS) += run-parts which +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DIFFUTILS) += cmp diff +# test1 removes the [ applet +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_COREUTILS) += \ + chmod tail printf uname touch echo od realpath sha512sum rmdir readlink \ + sync mknod du true nlink ln ls fold who logname chown dirname chgrp \ + basename uptime sha256sum uniq env rm expr wc pwd tee cat test chroot \ + nohup head sha1sum df dd nl seq truncate id cp mkfifo tty whoami factor \ + tr mkdir paste cksum sort stty shred md5sum hostid nproc install date \ + mv sleep yes cut false unlink test1 +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_FBSET) += fbset +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_KMOD_TOOLS) += lsmod rmmod modprobe insmod +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += blkid +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += dmesg +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += fdisk +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += flock +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += fstrim +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += hexdump +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += mkswap +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += readprofile +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += renice +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += setsid +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += swapoff +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += swapon +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_EJECT) += eject +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FALLOCATE) += fallocate +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FDFORMAT) += fdformat +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FSCK) += fsck +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_HWCLOCK) += hwclock +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_IPCRM) += ipcrm +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_IPCS) += ipcs +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_KILL) += kill +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LAST) += last +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOGGER) += logger +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOGIN) += login +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOSETUP) += losetup +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MESG) += mesg +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MORE) += more +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MOUNT) += mount umount +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MOUNTPOINT) += mountpoint +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_PIVOT_ROOT) += pivot_root +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SCHEDUTILS) += chrt +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SU) += su +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SULOGIN) += sulogin +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SWITCH_ROOT) += switch_root + +define BUSYBOX_DROP_CONFLICTING_APPLETS + $(foreach applet,$(BUSYBOX_DROP_APPLET_y),\ + $(call KCONFIG_DISABLE_OPT,CONFIG_$(call UPPERCASE,$(applet)),$(BUSYBOX_BUILD_CONFIG)) + ) +endef + # Telnet support define BUSYBOX_INSTALL_TELNET_SCRIPT if grep -q CONFIG_FEATURE_TELNETD_STANDALONE=y $(@D)/.config; then \ @@ -276,6 +333,7 @@ define BUSYBOX_KCONFIG_FIXUP_CMDS $(BUSYBOX_SET_SELINUX) $(BUSYBOX_SET_INDIVIDUAL_BINARIES) $(BUSYBOX_MUSL_TWEAKS) + $(BUSYBOX_DROP_CONFLICTING_APPLETS) endef define BUSYBOX_CONFIGURE_CMDS -- 2.14.3 ^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-13 13:01 ` [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages Thomas Petazzoni @ 2017-12-13 14:43 ` Baruch Siach 2017-12-14 5:18 ` Thomas Petazzoni 0 siblings, 1 reply; 26+ messages in thread From: Baruch Siach @ 2017-12-13 14:43 UTC (permalink / raw) To: buildroot Hi Thomas, On Wed, Dec 13, 2017 at 02:01:30PM +0100, Thomas Petazzoni wrote: > One of the requirement to implement per-package SDK is that a package > should not overwrite files installed by another package. The busybox > package creates such a situation with numerous other packages, because > it provides applets that are also provided as full-featured programs > in other packages. > > In order to avoid having other packages overwrite the Busybox applet > symbolic link with their own variant, this commit improves the logic > in busybox.mk to disable the applets when the corresponding > full-featured program will be installed by a different package. This means that a custom Busybox .config will automagically be transformed to something quite different. This is a significant change in behaviour. Can't we just move the symlink removal logic into busybox.mk instead? That would leave the busybox binary alone, and keep the magic .config transformations to minimum. baruch > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > --- > Note: the list is not complete, this is just a RFC version to find out > if that's the direction we want to take. > --- > package/busybox/busybox.mk | 58 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 58 insertions(+) > > diff --git a/package/busybox/busybox.mk b/package/busybox/busybox.mk > index 8b720b30b8..e51dbc4887 100644 > --- a/package/busybox/busybox.mk > +++ b/package/busybox/busybox.mk > @@ -250,6 +250,63 @@ endef > BUSYBOX_DEPENDENCIES += linux-pam > endif > > +# Avoid collision with other packages > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_BC) += dc > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_BINUTILS) += ar strings > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_CPIO) += cpio > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DCRON) += crond crontab > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DEBIANUTILS) += run-parts which > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_DIFFUTILS) += cmp diff > +# test1 removes the [ applet > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_COREUTILS) += \ > + chmod tail printf uname touch echo od realpath sha512sum rmdir readlink \ > + sync mknod du true nlink ln ls fold who logname chown dirname chgrp \ > + basename uptime sha256sum uniq env rm expr wc pwd tee cat test chroot \ > + nohup head sha1sum df dd nl seq truncate id cp mkfifo tty whoami factor \ > + tr mkdir paste cksum sort stty shred md5sum hostid nproc install date \ > + mv sleep yes cut false unlink test1 > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_FBSET) += fbset > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_KMOD_TOOLS) += lsmod rmmod modprobe insmod > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += blkid > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += dmesg > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += fdisk > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += flock > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += fstrim > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += hexdump > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += mkswap > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += readprofile > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += renice > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += setsid > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += swapoff > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_BINARIES) += swapon > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_EJECT) += eject > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FALLOCATE) += fallocate > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FDFORMAT) += fdformat > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_FSCK) += fsck > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_HWCLOCK) += hwclock > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_IPCRM) += ipcrm > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_IPCS) += ipcs > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_KILL) += kill > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LAST) += last > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOGGER) += logger > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOGIN) += login > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_LOSETUP) += losetup > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MESG) += mesg > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MORE) += more > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MOUNT) += mount umount > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_MOUNTPOINT) += mountpoint > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_PIVOT_ROOT) += pivot_root > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SCHEDUTILS) += chrt > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SU) += su > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SULOGIN) += sulogin > +BUSYBOX_DROP_APPLET_$(BR2_PACKAGE_UTIL_LINUX_SWITCH_ROOT) += switch_root > + > +define BUSYBOX_DROP_CONFLICTING_APPLETS > + $(foreach applet,$(BUSYBOX_DROP_APPLET_y),\ > + $(call KCONFIG_DISABLE_OPT,CONFIG_$(call UPPERCASE,$(applet)),$(BUSYBOX_BUILD_CONFIG)) > + ) > +endef > + > # Telnet support > define BUSYBOX_INSTALL_TELNET_SCRIPT > if grep -q CONFIG_FEATURE_TELNETD_STANDALONE=y $(@D)/.config; then \ > @@ -276,6 +333,7 @@ define BUSYBOX_KCONFIG_FIXUP_CMDS > $(BUSYBOX_SET_SELINUX) > $(BUSYBOX_SET_INDIVIDUAL_BINARIES) > $(BUSYBOX_MUSL_TWEAKS) > + $(BUSYBOX_DROP_CONFLICTING_APPLETS) > endef > > define BUSYBOX_CONFIGURE_CMDS -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-13 14:43 ` Baruch Siach @ 2017-12-14 5:18 ` Thomas Petazzoni 2017-12-14 6:58 ` Baruch Siach 0 siblings, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-14 5:18 UTC (permalink / raw) To: buildroot Hello, On Wed, 13 Dec 2017 16:43:05 +0200, Baruch Siach wrote: > This means that a custom Busybox .config will automagically be transformed to > something quite different. This is a significant change in behaviour. We already tweak the Busybox .config file quite significantly in busybox.mk, so it's not really new. But I agree that the scale of the change is very different. > Can't we just move the symlink removal logic into busybox.mk instead? > That would leave the busybox binary alone, and keep the magic .config > transformations to minimum. If I understand your proposal, we would not tweak the Busybox .config file, but instead add a post install target hook that would remove the symbolic links installed by Busybox "make install" when such symbolic links are going to conflict with full-blown versions of the corresponding applications, added by other packages. Is that correct ? If so, then I see two drawbacks with this approach. They are not unacceptable, but they are drawbacks: 1. We have to keep the optional dependency on Busybox in util-linux, coreutils, fbset, and all those packages that conflict with Busybox. Indeed, removing the symbolic link installed by Busybox only works if Busybox is installed first. Unless Busybox "make install" is careful enough to not install a given symlink if a file of the same name already exists. 2. The Busybox binary would contain lots of useless code, which simply can't be reached, except by calling explicitly "busybox cmd". So neither are unacceptable drawbacks, but they are drawbacks. If the general opinion is that we should do what you propose, I'm fine with implementing that. What do others think ? Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-14 5:18 ` Thomas Petazzoni @ 2017-12-14 6:58 ` Baruch Siach 2017-12-14 7:17 ` Thomas Petazzoni 2017-12-28 16:23 ` Yann E. MORIN 0 siblings, 2 replies; 26+ messages in thread From: Baruch Siach @ 2017-12-14 6:58 UTC (permalink / raw) To: buildroot Hi Thomas, On Thu, Dec 14, 2017 at 06:18:50AM +0100, Thomas Petazzoni wrote: > On Wed, 13 Dec 2017 16:43:05 +0200, Baruch Siach wrote: > > This means that a custom Busybox .config will automagically be transformed > > to something quite different. This is a significant change in behaviour. > > We already tweak the Busybox .config file quite significantly in > busybox.mk, so it's not really new. But I agree that the scale of the > change is very different. > > > Can't we just move the symlink removal logic into busybox.mk instead? > > That would leave the busybox binary alone, and keep the magic .config > > transformations to minimum. > > If I understand your proposal, we would not tweak the Busybox .config > file, but instead add a post install target hook that would remove the > symbolic links installed by Busybox "make install" when such symbolic > links are going to conflict with full-blown versions of the > corresponding applications, added by other packages. Is that correct ? > > If so, then I see two drawbacks with this approach. They are not > unacceptable, but they are drawbacks: > > 1. We have to keep the optional dependency on Busybox in util-linux, > coreutils, fbset, and all those packages that conflict with Busybox. > Indeed, removing the symbolic link installed by Busybox only works > if Busybox is installed first. Unless Busybox "make install" is > careful enough to not install a given symlink if a file of the > same name already exists. Quoting busybox.mk: # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any # full-blown versions of apps installed by other packages with sym/hard links. Unfortunately, the noclobber implementation is racy. So the alternative is to remove apps from the generated busybox.links file that install.sh uses as input. > 2. The Busybox binary would contain lots of useless code, which simply > can't be reached, except by calling explicitly "busybox cmd". This is not different than the situation we have now. The added useless code size is in most cases negligible compared to any one of the full-blown version. In the end it's the responsibility of the user to fine tune the target image size. > So neither are unacceptable drawbacks, but they are drawbacks. If the > general opinion is that we should do what you propose, I'm fine with > implementing that. > > What do others think ? baruch -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-14 6:58 ` Baruch Siach @ 2017-12-14 7:17 ` Thomas Petazzoni 2017-12-28 16:23 ` Yann E. MORIN 1 sibling, 0 replies; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-14 7:17 UTC (permalink / raw) To: buildroot Hello, On Thu, 14 Dec 2017 08:58:07 +0200, Baruch Siach wrote: > Quoting busybox.mk: > > # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any > # full-blown versions of apps installed by other packages with sym/hard links. > > Unfortunately, the noclobber implementation is racy. So the alternative is to > remove apps from the generated busybox.links file that install.sh uses as > input. The problem is that this busybox.links file gets generated as part of the "install" target. So unless we explicitly call "make busybox.links" in a pre-install hook, and then tweak it to remove the applets we don't want, I don't see how it could work. Can be done, but it starts to be a bit tricky :) > > 2. The Busybox binary would contain lots of useless code, which simply > > can't be reached, except by calling explicitly "busybox cmd". > > This is not different than the situation we have now. The added useless code > size is in most cases negligible compared to any one of the full-blown > version. In the end it's the responsibility of the user to fine tune the > target image size. I admit the size difference is minimal, and is clearly not the primary motivation for this. To me, it was merely a nice side effect. I measured a 70-80 KB reduction in the Busybox size when BR2_PACKAGE_COREUTILS=y, so it's clearly not much compared to how much disk space is consumed by the full-blown coreutils. Still, I find it weird that we build support in the binary for something that we don't expose in the target, while in terms of complexity, disabling them in the configuration is as simple (perhaps even more?) than preventing the symlinks from being created. But again, if it's the general consensus, I'm fine with it. As long as it helps things move forward towards per-package SDK and TLP support. It would be nice to hear from Peter, Arnout or Yann for example. Thanks for the feedback! Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-14 6:58 ` Baruch Siach 2017-12-14 7:17 ` Thomas Petazzoni @ 2017-12-28 16:23 ` Yann E. MORIN 2017-12-28 22:56 ` Yann E. MORIN 1 sibling, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2017-12-28 16:23 UTC (permalink / raw) To: buildroot Baruch, All, On 2017-12-14 08:58 +0200, Baruch Siach spake thusly: > Hi Thomas, > > On Thu, Dec 14, 2017 at 06:18:50AM +0100, Thomas Petazzoni wrote: > > On Wed, 13 Dec 2017 16:43:05 +0200, Baruch Siach wrote: > > > This means that a custom Busybox .config will automagically be transformed > > > to something quite different. This is a significant change in behaviour. > > > > We already tweak the Busybox .config file quite significantly in > > busybox.mk, so it's not really new. But I agree that the scale of the > > change is very different. > > > > > Can't we just move the symlink removal logic into busybox.mk instead? > > > That would leave the busybox binary alone, and keep the magic .config > > > transformations to minimum. > > > > If I understand your proposal, we would not tweak the Busybox .config > > file, but instead add a post install target hook that would remove the > > symbolic links installed by Busybox "make install" when such symbolic > > links are going to conflict with full-blown versions of the > > corresponding applications, added by other packages. Is that correct ? > > > > If so, then I see two drawbacks with this approach. They are not > > unacceptable, but they are drawbacks: > > > > 1. We have to keep the optional dependency on Busybox in util-linux, > > coreutils, fbset, and all those packages that conflict with Busybox. > > Indeed, removing the symbolic link installed by Busybox only works > > if Busybox is installed first. Unless Busybox "make install" is > > careful enough to not install a given symlink if a file of the > > same name already exists. > > Quoting busybox.mk: > > # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any > # full-blown versions of apps installed by other packages with sym/hard links. > > Unfortunately, the noclobber implementation is racy. How is it racy? At the moment we're gonna call it, no other package will be installing at the same time (in the location seen by busybox at least), so there should be no race. Regards, Yann E. MORIN. > So the alternative is to > remove apps from the generated busybox.links file that install.sh uses as > input. > > > 2. The Busybox binary would contain lots of useless code, which simply > > can't be reached, except by calling explicitly "busybox cmd". > > This is not different than the situation we have now. The added useless code > size is in most cases negligible compared to any one of the full-blown > version. In the end it's the responsibility of the user to fine tune the > target image size. > > > So neither are unacceptable drawbacks, but they are drawbacks. If the > > general opinion is that we should do what you propose, I'm fine with > > implementing that. > > > > What do others think ? > > baruch > > -- > http://baruch.siach.name/blog/ ~. .~ Tk Open Systems > =}------------------------------------------------ooO--U--Ooo------------{= > - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il - > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-28 16:23 ` Yann E. MORIN @ 2017-12-28 22:56 ` Yann E. MORIN 2017-12-29 5:59 ` Baruch Siach 0 siblings, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2017-12-28 22:56 UTC (permalink / raw) To: buildroot Baruch, Thomas, All, On 2017-12-28 17:23 +0100, Yann E. MORIN spake thusly: > On 2017-12-14 08:58 +0200, Baruch Siach spake thusly: [--SNIP--] > > Quoting busybox.mk: > > > > # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any > > # full-blown versions of apps installed by other packages with sym/hard links. > > > > Unfortunately, the noclobber implementation is racy. > > How is it racy? > > At the moment we're gonna call it, no other package will be installing > at the same time (in the location seen by busybox at least), so there > should be no race. Yet there were some issues with that part, so I've sent a small patch-series to busybox to fix the noclobber stuff: http://lists.busybox.net/pipermail/busybox/2017-December/086039.html So, with this, we should be able to: - invert the dependency chain, so thatr busybox now depends on all otehr packages that it may install applets of; - drop the big list of applets-to-disable in busybox, and rely on the install-noclobber install rule If upstream accepts that series, of course... Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-28 22:56 ` Yann E. MORIN @ 2017-12-29 5:59 ` Baruch Siach 2017-12-29 9:38 ` Yann E. MORIN 0 siblings, 1 reply; 26+ messages in thread From: Baruch Siach @ 2017-12-29 5:59 UTC (permalink / raw) To: buildroot Hi Yann, On Thu, Dec 28, 2017 at 11:56:39PM +0100, Yann E. MORIN wrote: > On 2017-12-28 17:23 +0100, Yann E. MORIN spake thusly: > > On 2017-12-14 08:58 +0200, Baruch Siach spake thusly: > [--SNIP--] > > > Quoting busybox.mk: > > > > > > # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any > > > # full-blown versions of apps installed by other packages with sym/hard links. > > > > > > Unfortunately, the noclobber implementation is racy. > > > > How is it racy? > > > > At the moment we're gonna call it, no other package will be installing > > at the same time (in the location seen by busybox at least), so there > > should be no race. > > Yet there were some issues with that part, so I've sent a small > patch-series to busybox to fix the noclobber stuff: > http://lists.busybox.net/pipermail/busybox/2017-December/086039.html > > So, with this, we should be able to: > > - invert the dependency chain, so thatr busybox now depends on all > otehr packages that it may install applets of; Why would we need a dependency at all? Either busybox builds before that other package (like we do today) in which case the other package overwrites the busybox symlink, or that other package builds before busybox, and then the noclobber option does the right thing. baruch > - drop the big list of applets-to-disable in busybox, and rely on the > install-noclobber install rule > > If upstream accepts that series, of course... -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 5:59 ` Baruch Siach @ 2017-12-29 9:38 ` Yann E. MORIN 2017-12-29 9:42 ` Thomas Petazzoni 2017-12-29 19:54 ` Trent Piepho 0 siblings, 2 replies; 26+ messages in thread From: Yann E. MORIN @ 2017-12-29 9:38 UTC (permalink / raw) To: buildroot Baruch, All, On 2017-12-29 07:59 +0200, Baruch Siach spake thusly: > On Thu, Dec 28, 2017 at 11:56:39PM +0100, Yann E. MORIN wrote: > > On 2017-12-28 17:23 +0100, Yann E. MORIN spake thusly: > > > On 2017-12-14 08:58 +0200, Baruch Siach spake thusly: > > [--SNIP--] > > > > Quoting busybox.mk: > > > > > > > > # Enable "noclobber" in install.sh, to prevent BusyBox from overwriting any > > > > # full-blown versions of apps installed by other packages with sym/hard links. > > > > > > > > Unfortunately, the noclobber implementation is racy. > > > > > > How is it racy? > > > > > > At the moment we're gonna call it, no other package will be installing > > > at the same time (in the location seen by busybox at least), so there > > > should be no race. > > > > Yet there were some issues with that part, so I've sent a small > > patch-series to busybox to fix the noclobber stuff: > > http://lists.busybox.net/pipermail/busybox/2017-December/086039.html > > > > So, with this, we should be able to: > > > > - invert the dependency chain, so thatr busybox now depends on all > > otehr packages that it may install applets of; > > Why would we need a dependency at all? Either busybox builds before that other > package (like we do today) in which case the other package overwrites the > busybox symlink, or that other package builds before busybox, and then the > noclobber option does the right thing. As was discussed during the DevDays in Pragues about top-level parallel build, we've concluded that we could not have two packages that touch the same file. When we are doing top-level parallel build, we no longer have any guarantee of the ordering, especially the order packages install in target/, so we loose the reproducibility that sequentiallity currently provides. So we have to ensure proper install ordering, so that we guarantee what files are in target/. So we either go with the current dependencies, added to all packages to ensure they get installed after busybox, or we add them to busybox so it gets installed after all the packages for which it may provide applets. I prefer the second option, because it concentrates the dependency chain in a single package, and it becomes very easy to write: BUSYBOX_DEPENDENCIES = \ $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ [...] Rather than duplicate the optional dependency on Busybox in all the packages (and as Thomas demonstrated, there are quite a few of them). I may even go further: we could even make busybox depend on *all* packages, irrespective on whether it provides applets for them or not, since busybiox is pretty fast to build and does build in parallel quite nicely anyway. That would nicely solve the issue. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 9:38 ` Yann E. MORIN @ 2017-12-29 9:42 ` Thomas Petazzoni 2017-12-29 9:52 ` Yann E. MORIN 2017-12-29 19:54 ` Trent Piepho 1 sibling, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-29 9:42 UTC (permalink / raw) To: buildroot Hello, On Fri, 29 Dec 2017 10:38:18 +0100, Yann E. MORIN wrote: > As was discussed during the DevDays in Pragues about top-level parallel > build, we've concluded that we could not have two packages that touch > the same file. > > When we are doing top-level parallel build, we no longer have any > guarantee of the ordering, especially the order packages install in > target/, so we loose the reproducibility that sequentiallity currently > provides. > > So we have to ensure proper install ordering, so that we guarantee what > files are in target/. > > So we either go with the current dependencies, added to all packages to > ensure they get installed after busybox, or we add them to busybox so it > gets installed after all the packages for which it may provide applets. > > I prefer the second option, because it concentrates the dependency chain > in a single package, and it becomes very easy to write: > > BUSYBOX_DEPENDENCIES = \ > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ > [...] > > Rather than duplicate the optional dependency on Busybox in all the > packages (and as Thomas demonstrated, there are quite a few of them). Seems like a good idea to me. > I may even go further: we could even make busybox depend on *all* > packages, irrespective on whether it provides applets for them or not, > since busybiox is pretty fast to build and does build in parallel quite > nicely anyway. That would nicely solve the issue. I don't really like this approach, I find it too brutal. Since we have the testing logic to validate that no package overwrites files from another package, I think the approach of having explicit dependencies in Busybox is good enough. Best regards, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 9:42 ` Thomas Petazzoni @ 2017-12-29 9:52 ` Yann E. MORIN 2017-12-29 9:55 ` Thomas Petazzoni 0 siblings, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2017-12-29 9:52 UTC (permalink / raw) To: buildroot Thomas, All, On 2017-12-29 10:42 +0100, Thomas Petazzoni spake thusly: > On Fri, 29 Dec 2017 10:38:18 +0100, Yann E. MORIN wrote: > > As was discussed during the DevDays in Pragues about top-level parallel > > build, we've concluded that we could not have two packages that touch > > the same file. > > > > When we are doing top-level parallel build, we no longer have any > > guarantee of the ordering, especially the order packages install in > > target/, so we loose the reproducibility that sequentiallity currently > > provides. > > > > So we have to ensure proper install ordering, so that we guarantee what > > files are in target/. > > > > So we either go with the current dependencies, added to all packages to > > ensure they get installed after busybox, or we add them to busybox so it > > gets installed after all the packages for which it may provide applets. > > > > I prefer the second option, because it concentrates the dependency chain > > in a single package, and it becomes very easy to write: > > > > BUSYBOX_DEPENDENCIES = \ > > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > > $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ > > [...] > > > > Rather than duplicate the optional dependency on Busybox in all the > > packages (and as Thomas demonstrated, there are quite a few of them). > > Seems like a good idea to me. For that, we'll need the three-patch series I submitted to Busybox: http://lists.busybox.net/pipermail/busybox/2017-December/086039.html > > I may even go further: we could even make busybox depend on *all* > > packages, irrespective on whether it provides applets for them or not, > > since busybiox is pretty fast to build and does build in parallel quite > > nicely anyway. That would nicely solve the issue. > > I don't really like this approach, I find it too brutal. Sorry, I was not advocating for this solution. What I meant was that, to simplify things even further we could do that. But I don't think we need that. > Since we have the testing logic to validate that no package overwrites > files from another package, I think the approach of having explicit > dependencies in Busybox is good enough. Once we turn the warniong into a hard error, yes. ;-) Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 9:52 ` Yann E. MORIN @ 2017-12-29 9:55 ` Thomas Petazzoni 2018-01-04 15:20 ` Yann E. MORIN 0 siblings, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-29 9:55 UTC (permalink / raw) To: buildroot Hello, On Fri, 29 Dec 2017 10:52:42 +0100, Yann E. MORIN wrote: > > > Rather than duplicate the optional dependency on Busybox in all the > > > packages (and as Thomas demonstrated, there are quite a few of them). > > > > Seems like a good idea to me. > > For that, we'll need the three-patch series I submitted to Busybox: > http://lists.busybox.net/pipermail/busybox/2017-December/086039.html Absolutely. Hopefully, upstream will give some feedback soon. > > I don't really like this approach, I find it too brutal. > > Sorry, I was not advocating for this solution. What I meant was that, to > simplify things even further we could do that. But I don't think we need > that. OK, then we agree :) > > Since we have the testing logic to validate that no package overwrites > > files from another package, I think the approach of having explicit > > dependencies in Busybox is good enough. > > Once we turn the warniong into a hard error, yes. ;-) In fact, I'd like to turn this into a hard error fairly soon. Or perhaps I could do some research in the autobuilder logs to identify the issues (other than Busybox) so that we try to fix them ahead of time. Best regards, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 9:55 ` Thomas Petazzoni @ 2018-01-04 15:20 ` Yann E. MORIN 2018-01-04 15:29 ` Thomas Petazzoni 0 siblings, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2018-01-04 15:20 UTC (permalink / raw) To: buildroot Thomas, All, On 2017-12-29 10:55 +0100, Thomas Petazzoni spake thusly: > On Fri, 29 Dec 2017 10:52:42 +0100, Yann E. MORIN wrote: > > > > Rather than duplicate the optional dependency on Busybox in all the > > > > packages (and as Thomas demonstrated, there are quite a few of them). > > > Seems like a good idea to me. > > For that, we'll need the three-patch series I submitted to Busybox: > > http://lists.busybox.net/pipermail/busybox/2017-December/086039.html > Absolutely. Hopefully, upstream will give some feedback soon. Upstream has merged those three patches now, so we can rely on a clean way to install without clobbering existing utils; we just have to use: make install-noclobber Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2018-01-04 15:20 ` Yann E. MORIN @ 2018-01-04 15:29 ` Thomas Petazzoni 2018-01-04 15:39 ` Yann E. MORIN 0 siblings, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2018-01-04 15:29 UTC (permalink / raw) To: buildroot Hello, On Thu, 4 Jan 2018 16:20:48 +0100, Yann E. MORIN wrote: > Upstream has merged those three patches now, so we can rely on a clean > way to install without clobbering existing utils; we just have to use: > make install-noclobber Great! So the idea is to have Busybox install its stuff *after* all the packages that install full-blown versions. Correct ? Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2018-01-04 15:29 ` Thomas Petazzoni @ 2018-01-04 15:39 ` Yann E. MORIN 0 siblings, 0 replies; 26+ messages in thread From: Yann E. MORIN @ 2018-01-04 15:39 UTC (permalink / raw) To: buildroot Thomas, All, On 2018-01-04 16:29 +0100, Thomas Petazzoni spake thusly: > On Thu, 4 Jan 2018 16:20:48 +0100, Yann E. MORIN wrote: > > Upstream has merged those three patches now, so we can rely on a clean > > way to install without clobbering existing utils; we just have to use: > > make install-noclobber > Great! So the idea is to have Busybox install its stuff *after* all the > packages that install full-blown versions. Correct ? Yes, that is what I understood from the discussion, we would just need to: 1. remove the conditional dependencies on busybox in all packages; 2. add a conditrional dependency in busybox for each package of which busybox may install an applet; 3. use the install-noclobber rule; 4. drop the BUSYBOX_NOCLOBBER_INSTALL hook. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 9:38 ` Yann E. MORIN 2017-12-29 9:42 ` Thomas Petazzoni @ 2017-12-29 19:54 ` Trent Piepho 2017-12-29 20:18 ` Yann E. MORIN 1 sibling, 1 reply; 26+ messages in thread From: Trent Piepho @ 2017-12-29 19:54 UTC (permalink / raw) To: buildroot On Fri, 2017-12-29 at 10:38 +0100, Yann E. MORIN wrote: > > I prefer the second option, because it concentrates the dependency chain > in a single package, and it becomes very easy to write: > > BUSYBOX_DEPENDENCIES = \ > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ > [...] Could one have a package variable specifically for install dependencies, like: BUSYBOX_INSTALL_DEPENDENCIES = coreutils util-linux ... It seems like the base pkg infra could automagically turn that into conditional dependencies, e.g. "coreutils" -> "(if $(BR2_PACKAGE_COREUTILS), coreutils)". And then append those to the package dependencies. So less boilerplate to write in the package file. But it would also allow the possibility to take advantage of knowing that coreutils is an install dependency rather than a build dependency. So it's allowed to build busybox and coreutils at the same time in parallel. The requirement is only that coreutils be installed to the target dir first. Usually installing is much faster than building, so one gets more parallelism this way. Of course this optimization is not necessary, but separating out the install deps now allows the possibility in the future, while being less boilerplate and more clear documentation now. ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 19:54 ` Trent Piepho @ 2017-12-29 20:18 ` Yann E. MORIN 2017-12-29 21:50 ` Trent Piepho 0 siblings, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2017-12-29 20:18 UTC (permalink / raw) To: buildroot Trent, All, On 2017-12-29 19:54 +0000, Trent Piepho spake thusly: > On Fri, 2017-12-29 at 10:38 +0100, Yann E. MORIN wrote: > > > > I prefer the second option, because it concentrates the dependency chain > > in a single package, and it becomes very easy to write: > > > > BUSYBOX_DEPENDENCIES = \ > > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > > $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ > > [...] > > Could one have a package variable specifically for install > dependencies, like: I think you already suggested so on IRC a while back, right? In fact, I do see the reasoning behind this, but I don;t think there is a big advantage for us, in Buildroot. In any case, we will want those dependencies to be installed befiore the package itself is installed, so we don't care that they were installed before the package is built. One case where that might be interesting is to test-build a package without building its install dependencies. But except for busybox, it would only be usefull for very few packages, because in the vast majority of cases, dependencies are build dependencies. So I don't think it warrants the extra complexity in the infra... > BUSYBOX_INSTALL_DEPENDENCIES = coreutils util-linux ... If we were to add support for install dependencies, we would want to treat them as the other dependencies and not do magical thigns like list all of them and defer to the infra to sort out those that are enabeld. In fact, that would go contrary to what we currently do for the existing dependencies: if a package lists a dependency but it is not enabled, we explicitly fail. So we'd want to have: BUSYBOX_INSTALL_DEPENDENCIES = \ $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ [...] Adn then... > It seems like the base pkg infra could automagically turn that into > conditional dependencies, e.g. "coreutils" -> "(if > $(BR2_PACKAGE_COREUTILS), coreutils)". And then append those to the > package dependencies. > > So less boilerplate to write in the package file. ... that would not diminish the "boilerplate" in the package's .mk file. But know that this is specifically just for busybox (and maybe a very few other packages), so that really does not warrant extra support in the infra, IMHO... Regards, Yann E. MORIN. > But it would also allow the possibility to take advantage of knowing > that coreutils is an install dependency rather than a build dependency. > So it's allowed to build busybox and coreutils at the same time in > parallel. The requirement is only that coreutils be installed to the > target dir first. Usually installing is much faster than building, so > one gets more parallelism this way. > > Of course this optimization is not necessary, but separating out the > install deps now allows the possibility in the future, while being less > boilerplate and more clear documentation now. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages 2017-12-29 20:18 ` Yann E. MORIN @ 2017-12-29 21:50 ` Trent Piepho 0 siblings, 0 replies; 26+ messages in thread From: Trent Piepho @ 2017-12-29 21:50 UTC (permalink / raw) To: buildroot On Fri, 2017-12-29 at 21:18 +0100, Yann E. MORIN wrote: > On 2017-12-29 19:54 +0000, Trent Piepho spake thusly: > > On Fri, 2017-12-29 at 10:38 +0100, Yann E. MORIN wrote: > > > > > > I prefer the second option, because it concentrates the dependency chain > > > in a single package, and it becomes very easy to write: > > > > > > BUSYBOX_DEPENDENCIES = \ > > > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > > > $(if $(BR2_PACKAGE_UTIL_LINUX),util-linux) \ > > > [...] > > > > Could one have a package variable specifically for install > > dependencies, like: > > I think you already suggested so on IRC a while back, right? Yes LTIB did this. LTIB was fancier about automatically taking into account changes in configuration to minimize rebuilding (disks were slower then!) and this helped. I don't think it's important w.r.t. what buildroot could do here. Buildroot is different software of course and I don't think LTIB is important w.r.t. what buildroot could do here. Rather, consider that at one time buildroot could not build packages in parallel and now it will. Just because something does not appear useful now does not mean it never will be. > If we were to add support for install dependencies, we would want to > treat them as the other dependencies and not do magical thigns like list > all of them and defer to the infra to sort out those that are enabeld. > > In fact, that would go contrary to what we currently do for the existing > dependencies: if a package lists a dependency but it is not enabled, we > explicitly fail. But it seems like this the exact opposite of what is being done?! If one uses: BUSYBOX_DEPENDENCIES += $(if $(BR2_PACKAGE_COREUTILS),coreutils) Then make busybox-show-depends does NOT list coreutils, because I have not enabled coreutils. Same with graph-depends, coreutils-show- rdepends, and so on. But if one does: BUSYBOX_INSTALL_DEPENDENCIES += coreutils Then it's possible to choose another policy. One could make show- depends only show the dependency if coreutils is enabled, just like now, but add a show-all-install-depends target that lists all possible dependencies, whether enabled or not. Maybe someone will want that?graph-depends could use a different color for the lines. But if the distinction is not made, then all these possibilities go away. It's not that complex either, one line in pkg-generic before setting FINAL_DEPENDENCIES $(2)_DEPENDENCIES += $$(foreach p,$$($(2)_INSTALL_DEPENDENCIES),$$(if $$(BR2_PACKAGE_$$(call UPPERCASE,$$(p))),$$(p))) ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 2/2] packages: drop no longer needed busybox dependencies 2017-12-13 13:01 [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Thomas Petazzoni 2017-12-13 13:01 ` [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages Thomas Petazzoni @ 2017-12-13 13:01 ` Thomas Petazzoni 2017-12-28 17:00 ` [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Yann E. MORIN 2 siblings, 0 replies; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-13 13:01 UTC (permalink / raw) To: buildroot Now that the busybox package is careful to not build and install applets that will conflict with the full-featured programs installed by various packages, we can drop the optional dependency those packages had on busybox. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- Note: the list of packages is not complete, this is just a RFC version to find out if that's the direction we want to take. --- package/bc/bc.mk | 5 ----- package/binutils/binutils.mk | 5 ----- package/coreutils/coreutils.mk | 6 ------ package/cpio/cpio.mk | 1 - package/dcron/dcron.mk | 5 ----- package/debianutils/debianutils.mk | 2 -- package/diffutils/diffutils.mk | 4 ---- package/fbset/fbset.mk | 5 ----- package/kmod/kmod.mk | 3 --- package/util-linux/util-linux.mk | 6 ------ 10 files changed, 42 deletions(-) diff --git a/package/bc/bc.mk b/package/bc/bc.mk index a6446cef66..ee0e43848b 100644 --- a/package/bc/bc.mk +++ b/package/bc/bc.mk @@ -11,9 +11,4 @@ BC_DEPENDENCIES = host-flex BC_LICENSE = GPL-2.0+, LGPL-2.1+ BC_LICENSE_FILES = COPYING COPYING.LIB -# Build after busybox so target ends up with bc's "dc" version -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -BC_DEPENDENCIES += busybox -endif - $(eval $(autotools-package)) diff --git a/package/binutils/binutils.mk b/package/binutils/binutils.mk index 473fcac181..8fb1a53437 100644 --- a/package/binutils/binutils.mk +++ b/package/binutils/binutils.mk @@ -84,11 +84,6 @@ ifeq ($(BR2_ARM_CPU_ARMV7M)$(BR2_OPTIMIZE_S),yy) BINUTILS_CONF_ENV += CFLAGS="$(TARGET_CFLAGS) -O2" endif -# Install binutils after busybox to prefer full-blown utilities -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -BINUTILS_DEPENDENCIES += busybox -endif - ifeq ($(BR2_PACKAGE_ZLIB),y) BINUTILS_DEPENDENCIES += zlib endif diff --git a/package/coreutils/coreutils.mk b/package/coreutils/coreutils.mk index 6a8a31b061..3f41e6f1a4 100644 --- a/package/coreutils/coreutils.mk +++ b/package/coreutils/coreutils.mk @@ -59,12 +59,6 @@ COREUTILS_BIN_PROGS = cat chgrp chmod chown cp date dd df dir echo false \ ln ls mkdir mknod mv pwd rm rmdir vdir sleep stty sync touch true \ uname join -# If both coreutils and busybox are selected, make certain coreutils -# wins the fight over who gets to have their utils actually installed. -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -COREUTILS_DEPENDENCIES = busybox -endif - ifeq ($(BR2_PACKAGE_ACL),y) COREUTILS_DEPENDENCIES += acl else diff --git a/package/cpio/cpio.mk b/package/cpio/cpio.mk index 7e8549dbd8..9a09279572 100644 --- a/package/cpio/cpio.mk +++ b/package/cpio/cpio.mk @@ -7,7 +7,6 @@ CPIO_VERSION = 2.12 CPIO_SITE = $(BR2_GNU_MIRROR)/cpio CPIO_CONF_OPTS = --bindir=/bin -CPIO_DEPENDENCIES = $(if $(BR2_PACKAGE_BUSYBOX),busybox) CPIO_LICENSE = GPL-3.0+ CPIO_LICENSE_FILES = COPYING diff --git a/package/dcron/dcron.mk b/package/dcron/dcron.mk index 7e81b63803..2ee0709af5 100644 --- a/package/dcron/dcron.mk +++ b/package/dcron/dcron.mk @@ -9,11 +9,6 @@ DCRON_SITE = http://www.jimpryor.net/linux/releases # The source code does not specify the version of the GPL that is used. DCRON_LICENSE = GPL -# Overwrite cron-related Busybox commands if available -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -DCRON_DEPENDENCIES = busybox -endif - define DCRON_BUILD_CMDS $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(TARGET_CONFIGURE_OPTS) endef diff --git a/package/debianutils/debianutils.mk b/package/debianutils/debianutils.mk index 86f201da1c..189d983732 100644 --- a/package/debianutils/debianutils.mk +++ b/package/debianutils/debianutils.mk @@ -8,8 +8,6 @@ DEBIANUTILS_VERSION = 4.8.1 DEBIANUTILS_SOURCE = debianutils_$(DEBIANUTILS_VERSION).tar.xz DEBIANUTILS_SITE = http://snapshot.debian.org/archive/debian/20161118T033019Z/pool/main/d/debianutils DEBIANUTILS_CONF_OPTS = --exec-prefix=/ -# Make sure we override the busybox tools, such as which -DEBIANUTILS_DEPENDENCIES = $(if $(BR2_PACKAGE_BUSYBOX),busybox) DEBIANUTILS_LICENSE = GPL-2.0+, SMAIL (savelog) DEBIANUTILS_LICENSE_FILES = debian/copyright diff --git a/package/diffutils/diffutils.mk b/package/diffutils/diffutils.mk index 79785d6f01..9a41c985df 100644 --- a/package/diffutils/diffutils.mk +++ b/package/diffutils/diffutils.mk @@ -17,8 +17,4 @@ ifeq ($(BR2_TOOLCHAIN_USES_GLIBC),y) DIFFUTILS_CONF_ENV += gl_cv_func_getopt_gnu=yes endif -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -DIFFUTILS_DEPENDENCIES += busybox -endif - $(eval $(autotools-package)) diff --git a/package/fbset/fbset.mk b/package/fbset/fbset.mk index d6ee104a22..787b4e1dca 100644 --- a/package/fbset/fbset.mk +++ b/package/fbset/fbset.mk @@ -9,11 +9,6 @@ FBSET_SITE = http://users.telenet.be/geertu/Linux/fbdev FBSET_DEPENDENCIES = host-bison host-flex FBSET_LICENSE = GPL-2.0 -# Make sure full fbset wins over busybox fbset -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -FBSET_DEPENDENCIES += busybox -endif - define FBSET_BUILD_CMDS $(MAKE1) $(TARGET_CONFIGURE_OPTS) -C $(@D) endef diff --git a/package/kmod/kmod.mk b/package/kmod/kmod.mk index bd676108aa..0b67460a13 100644 --- a/package/kmod/kmod.mk +++ b/package/kmod/kmod.mk @@ -49,9 +49,6 @@ ifeq ($(BR2_PACKAGE_KMOD_TOOLS),y) KMOD_LICENSE := $(KMOD_LICENSE), GPL-2.0+ (tools) KMOD_LICENSE_FILES += COPYING -# take precedence over busybox implementation -KMOD_DEPENDENCIES += $(if $(BR2_PACKAGE_BUSYBOX),busybox) - # /sbin is really /usr/sbin with merged /usr, so adjust relative symlink ifeq ($(BR2_ROOTFS_MERGED_USR),y) KMOD_BIN_PATH = ../bin/kmod diff --git a/package/util-linux/util-linux.mk b/package/util-linux/util-linux.mk index 31a3fe47ab..16137f4791 100644 --- a/package/util-linux/util-linux.mk +++ b/package/util-linux/util-linux.mk @@ -35,12 +35,6 @@ HOST_UTIL_LINUX_DEPENDENCIES = host-pkgconf # We also don't want the host-python dependency HOST_UTIL_LINUX_CONF_OPTS = --without-python -# If both util-linux and busybox are selected, make certain util-linux -# wins the fight over who gets to have their utils actually installed -ifeq ($(BR2_PACKAGE_BUSYBOX),y) -UTIL_LINUX_DEPENDENCIES += busybox -endif - # Prevent the installation from attempting to move shared libraries from # ${usrlib_execdir} (/usr/lib) to ${libdir} (/lib), since both paths are # the same when merged usr is in use. -- 2.14.3 ^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox 2017-12-13 13:01 [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Thomas Petazzoni 2017-12-13 13:01 ` [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages Thomas Petazzoni 2017-12-13 13:01 ` [Buildroot] [RFC 2/2] packages: drop no longer needed busybox dependencies Thomas Petazzoni @ 2017-12-28 17:00 ` Yann E. MORIN 2017-12-28 17:04 ` Thomas Petazzoni 2 siblings, 1 reply; 26+ messages in thread From: Yann E. MORIN @ 2017-12-28 17:00 UTC (permalink / raw) To: buildroot Thomas, All, On 2017-12-13 14:01 +0100, Thomas Petazzoni spake thusly: > As we discussed during the Prague Buildroot Developers meeting, in > order to implement per-package SDK, we need to ensure that no package > overwrites files installed by another package. > > This RFC series is an attempt at solving this problem for Busybox. I > have not fixed all packages yet: since it is a very boring task to do, > I wanted to first get some feedback on whether the approach looks > reasonable or not. > > If the feedback is positive, I'll go ahead and submit proper patches > that fix all packages that conflict with Busybox. As I previously said on IRC: I do not much like the big list we will now have to maintain; that's sad... However, I like the fact that we can get rid of the many dependencies in so many packages here and there. :-) What I would have suggested, though, is to do what Baruch hinted at: use the noclobber install of Busybox, and then have Busybox depend on all the packages it provides applets for: BUSYBOX_DEPENDENCIES = \ $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ $(if $(BR2_PACKAGE_util_LINUX),util-linux) \ etc... But unfortunately, the noclobber install option is not usable: - first, there is no way to cause a noclobber install; - second, the noclobber is not accounted for in the case shell wrappers are used. So, I'm afraid we don't have much choice but to do as your series does... Regards, Yann E. MORIN. > Thanks for your feedback! > > Thomas > > Thomas Petazzoni (2): > busybox: avoid conflict with other packages > packages: drop no longer needed busybox dependencies > > package/bc/bc.mk | 5 ---- > package/binutils/binutils.mk | 5 ---- > package/busybox/busybox.mk | 58 ++++++++++++++++++++++++++++++++++++++ > package/coreutils/coreutils.mk | 6 ---- > package/cpio/cpio.mk | 1 - > package/dcron/dcron.mk | 5 ---- > package/debianutils/debianutils.mk | 2 -- > package/diffutils/diffutils.mk | 4 --- > package/fbset/fbset.mk | 5 ---- > package/kmod/kmod.mk | 3 -- > package/util-linux/util-linux.mk | 6 ---- > 11 files changed, 58 insertions(+), 42 deletions(-) > > -- > 2.14.3 > > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox 2017-12-28 17:00 ` [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Yann E. MORIN @ 2017-12-28 17:04 ` Thomas Petazzoni 2017-12-28 17:20 ` Yann E. MORIN 0 siblings, 1 reply; 26+ messages in thread From: Thomas Petazzoni @ 2017-12-28 17:04 UTC (permalink / raw) To: buildroot Hello, Thanks for the feedback. On Thu, 28 Dec 2017 18:00:17 +0100, Yann E. MORIN wrote: > > This RFC series is an attempt at solving this problem for Busybox. I > > have not fixed all packages yet: since it is a very boring task to do, > > I wanted to first get some feedback on whether the approach looks > > reasonable or not. > > > > If the feedback is positive, I'll go ahead and submit proper patches > > that fix all packages that conflict with Busybox. > > As I previously said on IRC: I do not much like the big list we will now > have to maintain; that's sad... Even though I agree it's not nice to maintain, I think it's unavoidable if we want to solve the overwriting issue. > However, I like the fact that we can get rid of the many dependencies in > so many packages here and there. :-) > > What I would have suggested, though, is to do what Baruch hinted at: use > the noclobber install of Busybox, and then have Busybox depend on all > the packages it provides applets for: > > BUSYBOX_DEPENDENCIES = \ > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > $(if $(BR2_PACKAGE_util_LINUX),util-linux) \ > etc... > > But unfortunately, the noclobber install option is not usable: > > - first, there is no way to cause a noclobber install; > > - second, the noclobber is not accounted for in the case shell > wrappers are used. > > So, I'm afraid we don't have much choice but to do as your series > does... There is however one remaining debate: my patch series tweaks the Busybox configuration to not build the support for applets for which the functionality is provided by a full-featured program. But Baruch didn't like this tweaking of the Busybox configuration, and would prefer to not install the symlinks, and leave the Busybox configuration unchanged (which means we have lots of Busybox applets built into Busybox that are not really used on the target). Do you have an opinion on this specific topic ? Another question is whether we want to have this logic centralized in busybox.mk, or spread into the packages that provide the full-featured variants of the applets ? The latter may have some variable definition ordering issues though. Thanks, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux and Kernel engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox 2017-12-28 17:04 ` Thomas Petazzoni @ 2017-12-28 17:20 ` Yann E. MORIN [not found] ` <CANQCQpZ-qO6v+K4kdqmAEdk2+Dk1Yca1fBqyNwfAjau=50cY7A@mail.gmail.com> 2017-12-28 18:01 ` Baruch Siach 0 siblings, 2 replies; 26+ messages in thread From: Yann E. MORIN @ 2017-12-28 17:20 UTC (permalink / raw) To: buildroot Thomas, All, On 2017-12-28 18:04 +0100, Thomas Petazzoni spake thusly: > Hello, > > Thanks for the feedback. > > On Thu, 28 Dec 2017 18:00:17 +0100, Yann E. MORIN wrote: > > > > This RFC series is an attempt at solving this problem for Busybox. I > > > have not fixed all packages yet: since it is a very boring task to do, > > > I wanted to first get some feedback on whether the approach looks > > > reasonable or not. > > > > > > If the feedback is positive, I'll go ahead and submit proper patches > > > that fix all packages that conflict with Busybox. > > > > As I previously said on IRC: I do not much like the big list we will now > > have to maintain; that's sad... > > Even though I agree it's not nice to maintain, I think it's unavoidable > if we want to solve the overwriting issue. So that we are on the same line: I do agree with the underlying reason for the change, yes. I'm just trying to see if there are better ways to go with that. > > However, I like the fact that we can get rid of the many dependencies in > > so many packages here and there. :-) > > > > What I would have suggested, though, is to do what Baruch hinted at: use > > the noclobber install of Busybox, and then have Busybox depend on all > > the packages it provides applets for: > > > > BUSYBOX_DEPENDENCIES = \ > > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > > $(if $(BR2_PACKAGE_util_LINUX),util-linux) \ > > etc... > > > > But unfortunately, the noclobber install option is not usable: > > > > - first, there is no way to cause a noclobber install; > > > > - second, the noclobber is not accounted for in the case shell > > wrappers are used. > > > > So, I'm afraid we don't have much choice but to do as your series > > does... > > There is however one remaining debate: my patch series tweaks the > Busybox configuration to not build the support for applets for which > the functionality is provided by a full-featured program. But Baruch > didn't like this tweaking of the Busybox configuration, and would > prefer to not install the symlinks, and leave the Busybox configuration > unchanged (which means we have lots of Busybox applets built into > Busybox that are not really used on the target). > > Do you have an opinion on this specific topic ? I prefer they be explicitly disabled as you did. First, because this is a security issue that there is dead code: these applets are still usable by calling 'busybox foo' for example, and that is a security issue. Second, yes it gains a bit of space. But that is not so compelling, becasue if you already have the big buddies enabled, a few kB lost inBusybox is not that much of a burden anyway... > Another question is whether we want to have this logic centralized in > busybox.mk, or spread into the packages that provide the full-featured > variants of the applets ? The latter may have some variable definition > ordering issues though. Please leave it centralised in busybox.mk. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
[parent not found: <CANQCQpZ-qO6v+K4kdqmAEdk2+Dk1Yca1fBqyNwfAjau=50cY7A@mail.gmail.com>]
[parent not found: <CANQCQpYmpCKopmh_5yYV74kOyezJSCLxp6T1mUiqnocHLZV92A@mail.gmail.com>]
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox [not found] ` <CANQCQpYmpCKopmh_5yYV74kOyezJSCLxp6T1mUiqnocHLZV92A@mail.gmail.com> @ 2017-12-28 17:36 ` Matthew Weber 0 siblings, 0 replies; 26+ messages in thread From: Matthew Weber @ 2017-12-28 17:36 UTC (permalink / raw) To: buildroot Thomas, All On Dec 28, 2017 11:20 AM, "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: Thomas, All, [Snip] I prefer they be explicitly disabled as you did. First, because this is a security issue that there is dead code: these applets are still usable by calling 'busybox foo' for example, and that is a security issue. Second, yes it gains a bit of space. But that is not so compelling, becasue if you already have the big buddies enabled, a few kB lost inBusybox is not that much of a burden anyway... When it comes to maintaining a custom BusyBox config (which could now be dynamically updated), maybe we should output a warning when we detect that we change a config value so the user can know to update their customizations. Unsure if this is best as a altered.configs file or just printing it out..... (Pardon the html response, no computer this week) Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20171228/32c0871e/attachment.html> ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox 2017-12-28 17:20 ` Yann E. MORIN [not found] ` <CANQCQpZ-qO6v+K4kdqmAEdk2+Dk1Yca1fBqyNwfAjau=50cY7A@mail.gmail.com> @ 2017-12-28 18:01 ` Baruch Siach 2017-12-28 19:11 ` Yann E. MORIN 1 sibling, 1 reply; 26+ messages in thread From: Baruch Siach @ 2017-12-28 18:01 UTC (permalink / raw) To: buildroot Hi Yann, On Thu, Dec 28, 2017 at 06:20:30PM +0100, Yann E. MORIN wrote: > On 2017-12-28 18:04 +0100, Thomas Petazzoni spake thusly: > > Thanks for the feedback. > > > > On Thu, 28 Dec 2017 18:00:17 +0100, Yann E. MORIN wrote: > > > > > > This RFC series is an attempt at solving this problem for Busybox. I > > > > have not fixed all packages yet: since it is a very boring task to do, > > > > I wanted to first get some feedback on whether the approach looks > > > > reasonable or not. > > > > > > > > If the feedback is positive, I'll go ahead and submit proper patches > > > > that fix all packages that conflict with Busybox. > > > > > > As I previously said on IRC: I do not much like the big list we will now > > > have to maintain; that's sad... > > > > Even though I agree it's not nice to maintain, I think it's unavoidable > > if we want to solve the overwriting issue. > > So that we are on the same line: I do agree with the underlying reason > for the change, yes. I'm just trying to see if there are better ways to > go with that. > > > > However, I like the fact that we can get rid of the many dependencies in > > > so many packages here and there. :-) > > > > > > What I would have suggested, though, is to do what Baruch hinted at: use > > > the noclobber install of Busybox, and then have Busybox depend on all > > > the packages it provides applets for: > > > > > > BUSYBOX_DEPENDENCIES = \ > > > $(if $(BR2_PACKAGE_COREUTILS),coreutils) \ > > > $(if $(BR2_PACKAGE_util_LINUX),util-linux) \ > > > etc... > > > > > > But unfortunately, the noclobber install option is not usable: > > > > > > - first, there is no way to cause a noclobber install; > > > > > > - second, the noclobber is not accounted for in the case shell > > > wrappers are used. > > > > > > So, I'm afraid we don't have much choice but to do as your series > > > does... > > > > There is however one remaining debate: my patch series tweaks the > > Busybox configuration to not build the support for applets for which > > the functionality is provided by a full-featured program. But Baruch > > didn't like this tweaking of the Busybox configuration, and would > > prefer to not install the symlinks, and leave the Busybox configuration > > unchanged (which means we have lots of Busybox applets built into > > Busybox that are not really used on the target). > > > > Do you have an opinion on this specific topic ? > > I prefer they be explicitly disabled as you did. As I said earlier, I don't like this heavy handed modification of the Busybox configuration. I'm not sure how robust this solution would be, as there might be unintended consequences. Kconfig 'select' and 'depends' contribute to that. I prefer to keep config modifications to minimum. > First, because this is a security issue that there is dead code: these > applets are still usable by calling 'busybox foo' for example, and that > is a security issue. That is no different than the current behaviour. > Second, yes it gains a bit of space. But that is not so compelling, > becasue if you already have the big buddies enabled, a few kB lost > inBusybox is not that much of a burden anyway... I agree. > > Another question is whether we want to have this logic centralized in > > busybox.mk, or spread into the packages that provide the full-featured > > variants of the applets ? The latter may have some variable definition > > ordering issues though. > > Please leave it centralised in busybox.mk. I agree here as well. baruch -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [RFC 0/2] Handle conflicting files with Busybox 2017-12-28 18:01 ` Baruch Siach @ 2017-12-28 19:11 ` Yann E. MORIN 0 siblings, 0 replies; 26+ messages in thread From: Yann E. MORIN @ 2017-12-28 19:11 UTC (permalink / raw) To: buildroot Baruch, Thomas, All, On 2017-12-28 20:01 +0200, Baruch Siach spake thusly: > On Thu, Dec 28, 2017 at 06:20:30PM +0100, Yann E. MORIN wrote: > > On 2017-12-28 18:04 +0100, Thomas Petazzoni spake thusly: [--SNIP--] > > > There is however one remaining debate: my patch series tweaks the > > > Busybox configuration to not build the support for applets for which > > > the functionality is provided by a full-featured program. But Baruch > > > didn't like this tweaking of the Busybox configuration, and would > > > prefer to not install the symlinks, and leave the Busybox configuration > > > unchanged (which means we have lots of Busybox applets built into > > > Busybox that are not really used on the target). > > > > > > Do you have an opinion on this specific topic ? > > > > I prefer they be explicitly disabled as you did. > > As I said earlier, I don't like this heavy handed modification of the Busybox > configuration. I'm not sure how robust this solution would be, as there might > be unintended consequences. Kconfig 'select' and 'depends' contribute to that. > I prefer to keep config modifications to minimum. Agreed that the 'select' things in busybox' Kconfig could be an issue. But do we have applets that get select-ed by another? In current busybox, there is no applet that is select-ed: $ git grep -E 'config:[[:space:]]+\<select ' |cut -d : -f 2- |sort -u //config:select FEATURE_BZIP2_DECOMPRESS //config:se lect FEATURE_GZIP_DECOMPRESS //config:select FEATURE_IP_ADDRESS //conf ig:select FEATURE_IP_LINK //config:select FEATURE_IP_NEIGH //config: select FEATURE_IP_ROUTE //config:select FEATURE_IP_RULE //config:sele ct FEATURE_IP_TUNNEL //config:select FEATURE_SEAMLESS_GZ //config:sel ect FEATURE_SYSLOG //config:select LONG_OPTS //config:select PLATFORM _LINUX //config:select PLATFORM_LINUX # statfs() //config:select PLAT FORM_LINUX #sysinfo() //config:select TLS //config:select VOLUMEID None of those are actual applets. But OK, there is no guarantee that this will continue to be the case. As an aside, we currently disable a bunch of Busybox options, of which only three applets: swapon, swapoff, ash. > > First, because this is a security issue that there is dead code: these > > applets are still usable by calling 'busybox foo' for example, and that > > is a security issue. > > That is no different than the current behaviour. Indeed not really. But it's not because we're doing something bad today, that we should pursue in this direction! ;-) In the end, my preference is still to do as Thomas proposed... Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2018-01-04 15:39 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-12-13 13:01 [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Thomas Petazzoni
2017-12-13 13:01 ` [Buildroot] [RFC 1/2] busybox: avoid conflict with other packages Thomas Petazzoni
2017-12-13 14:43 ` Baruch Siach
2017-12-14 5:18 ` Thomas Petazzoni
2017-12-14 6:58 ` Baruch Siach
2017-12-14 7:17 ` Thomas Petazzoni
2017-12-28 16:23 ` Yann E. MORIN
2017-12-28 22:56 ` Yann E. MORIN
2017-12-29 5:59 ` Baruch Siach
2017-12-29 9:38 ` Yann E. MORIN
2017-12-29 9:42 ` Thomas Petazzoni
2017-12-29 9:52 ` Yann E. MORIN
2017-12-29 9:55 ` Thomas Petazzoni
2018-01-04 15:20 ` Yann E. MORIN
2018-01-04 15:29 ` Thomas Petazzoni
2018-01-04 15:39 ` Yann E. MORIN
2017-12-29 19:54 ` Trent Piepho
2017-12-29 20:18 ` Yann E. MORIN
2017-12-29 21:50 ` Trent Piepho
2017-12-13 13:01 ` [Buildroot] [RFC 2/2] packages: drop no longer needed busybox dependencies Thomas Petazzoni
2017-12-28 17:00 ` [Buildroot] [RFC 0/2] Handle conflicting files with Busybox Yann E. MORIN
2017-12-28 17:04 ` Thomas Petazzoni
2017-12-28 17:20 ` Yann E. MORIN
[not found] ` <CANQCQpZ-qO6v+K4kdqmAEdk2+Dk1Yca1fBqyNwfAjau=50cY7A@mail.gmail.com>
[not found] ` <CANQCQpYmpCKopmh_5yYV74kOyezJSCLxp6T1mUiqnocHLZV92A@mail.gmail.com>
2017-12-28 17:36 ` Matthew Weber
2017-12-28 18:01 ` Baruch Siach
2017-12-28 19:11 ` Yann E. MORIN
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.